1 

Fluctuation relations: a pedagogical overview 

Richard Spinney and Ian Ford 

Department of Physics and Astronomy and London Centre for Nanotechnology, 
University College London, Gower Street, London WC1E 6BT, U.K^] 

1.1 

Preliminaries 

Ours is a harsh and unforgiving universe, and not just in the little matters that conspire 
against us. Its complicated rules of evolution seem unfairly biased against those who 
seek to predict the future. Of course, if the rules were simple, then there might be no 
universe of any complexity worth considering. Perhaps richness of behaviour only 
emerges because each component of the universe interacts with many others, and in 
ways that are very sensitive to details: this is the harsh and unforgiving nature. In 
order to predict the future, we have to take into account all the connections between 
the components, since they might be crucial to the evolution, and furthermore, we 
need to know everything about the present in order to predict the future: both of these 
requirements are in most cases impossible. Estimates and guesses are not enough: 
unforgiving sensitivity to the detail very soon leads to loss of predictability. We see 
this in the workings of a weather system. The approximations that meteorological 
services make in order to fill gaps in understanding, or initial data, eventually make 
the forecasts inaccurate. 

So a description of the dynamics of a complex system is likely to be incomplete 
and we have to accept that predictions will be uncertain. If we are careful in the 
modelling of the system, the uncertainty will grow only slowly. If we are sloppy in 
our model building or initial data collection, it will grow quickly. We might expect 
the predictions of any incomplete model to tend towards a state of general ignorance, 
whereby we cannot be sure about anything: rain, snow, heatwave or hurricane. We 
must expect there to be a spread, or fluctuations, in the outcomes of such a model. 

This discussion of the growth of uncertainty in predictions has a bearing on another 
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matter: the apparent irreversibility of all but the most simple physical processes. This 
refers to our inability to drive a system exactly backwards by reversing the external 
forces that guide its evolution. Consider the mechanical work required to compress 
a gas by a piston in a cylinder. We might hope to see the expended energy returned 
when we stop pushing and allow the gas to drive the piston all the way back to the 
starting point: but not all will be returned. The system seems to mislay some energy 
to the benefit of the wider environment. This is the familiar process of friction. The 
one-way dissipation of energy during mechanical processing is an example of the 
famous second law of thermodynamics. But the process is actually rather mysterious: 
what about the underlying reversibility of Newton's equations of motion? Why is the 
leakage of energy one-way? 

We might suspect that a failure to engineer the exact reversal of a compression 
is simply a consequence of a lack of control over all components of the gas and its 
environment: the difficulty in setting things up properly for the return leg implies the 
virtual impossibility of retracing the behaviour. So we might not expect to be able to 
retrace exactly. But why do we not sometimes see 'antifriction' ? A clue might be seen 
in the relative size and complexity of the system and its environment. The smaller 
system is likely to evolve in a more complicated fashion as a result of the coupling, 
whilst we might expect the larger environment to be much less affected. There is a 
disparity in the effect of the coupling on each participant, and it is believed that this 
is responsible for the apparent one-way nature of friction. It is possible to implement 
these ideas by modelling the behaviour of a system using uncertain, or stochastic 
dynamics. The probability of observing a reversal of the behaviour on the return leg 
can be calculated explicitly and it turns out that the difference between probabilities 
of observing a particular compression and seeing its reverse on the return leg leads to 
a measure of the irreversibility of natural processes. The second law is then a rather 
simple consequence of the dynamics. A similar asymmetric treatment of the effect 
on a system of coupling to a large environment is possible using deterministic and 
reversible non-linear dynamics. In both cases, Loschmidt's paradox, the apparent 
breakage of time reversal symmetry for thermally constrained systems, is evaded, 
though for different reasons. 

This chapter describes the so-called fluctuation relations, or theorems [ 1 2 3]|4]|5), 
that emerge from the analysis of a physical system interacting with its environment, 
and which provide the structure that leads to the conclusion just outlined. They can 
quantify unexpected outcomes in terms of the expected. They apply on microscopic 
as well as macroscopic scales, and indeed their consequences are most apparent when 
applied to small systems. They can be derived on the basis of a rather natural measure 
of irreversibility, just alluded to, that offers an interpretation of the second law and 
the associated concept of entropy production. The dynamical rules that control the 
universe might seem harsh and unforgiving, but they can also be charitable, and from 
them have emerged fluctuation relations that seem to provide a better understanding 
of entropy, uncertainty, and the limits of predictability. 

This chapter is structured as follows. In order to provide a context for the fluctua- 
tion relations suitable for newcomers to the field we begin with a brief summary of 
thermodynamic irreversibility, and then describe how stochastic dynamics might be 



modelled. We use a framework based on stochastic, rather than deterministic dynam- 
ics since developing both themes here might not provide the most succinct pedagog- 
ical introduction. Nevertheless, we refer to the deterministic framework briefly later 
on, to emphasise its equivalence. We discuss the identification of entropy production 
with the degree of departure from dynamical reversibility, and then take a careful 
look at the developments that follow, which include the various fluctuation relations, 
and consider how the second law might not operate as we expect. We illustrate the 
fluctuation relations using simple analytical models, as an aid to understanding. We 
conclude with some final remarks, but the broader implications are to be found else- 
where in this book, for which we hope this chapter will act as a helpful background. 

1.2 

Entropy and the second law 

Ignorance and uncertainty has never been an unusual state of affairs in human per- 
ception. In mechanics, Newton's laws of motion provided tools that seemed to dis- 
pel some of the haze: here were mathematical models that enabled the future to be 
foretold! They inspired attempts to predict future behaviour in other fields, particu- 
larly in thermodynamics, the study of systems through which matter and energy can 
flow. The particular focus in the early days of the field was the heat engine, a device 
whereby fuel, and the heat it can generate, can be converted into mechanical work. 
Its operation was discovered to produce a quantity called entropy, that could charac- 
terise the efficiency with which energy in the fuel could be converted into motion. 
Indeed entropy seemed to be generated whenever heat or matter flowed. The second 
law of thermodynamics famously states that the total entropy of the evolving universe 
is always increasing. But this statement still attracts discussion, more than 150 years 
after its introduction. We do not debate the meaning of Newton's second law any 
more, so why is the second law of thermodynamics so controversial? 

Well, it is hard to understand how there can be a physical quantity that never de- 
creases. Such a statement demands the breakage of the principle of time reversal 
symmetry, a difficulty referred to as Loschmidt's paradox. Newton's equations of 
motion do not specify a preferred direction in which time evolves. Time is a coordi- 
nate in a description of the universe and it is just convention that real world events 
take place while this coordinate increases. Given that we cannot actually run time 
backwards, we can demonstrate this symmetry in the following way. A sequence of 
events that takes place according to time reversal symmetric equations can be invert- 
ed by instantaneously reversing all the velocities of all the participating components 
and then proceeding forward in time once again, suitably reversing any external pro- 
tocol of driving forces, if necessary. The point is that any evolution can be imagined 
in reverse, according to Newton. We therefore don't expect to observe any quantity 
that only ever increases with time. This is the essence of Loschmidt's objection to 
Boltzmann's |6| mechanical interpretation of the second law. 

But nobody has been able to initiate a heat engine such that it sucks exhaust gases 
back into its furnace and combines them into fuel. The denial of such a spectacle is 
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empirical evidence for the operation of the second law, but it is also an expression of 
Loschmidt's paradox. Time reversal symmetry is broken by the apparent illegality of 
entropy-consuming processes, and that seems unacceptable. Perhaps we should not 
blindly accept the second law in the sense that has traditionally been ascribed to it. Or 
perhaps there is something deeper going on. Furthermore, a law that only specifies 
the sign of a rate of change sounds rather incomplete. 

But what has emerged in the last two decades or so is the realisation that Newton's 
laws of motion, when supplemented by the acceptance of uncertainty in the way sys- 
tems behave, brought about by roughly specified interactions with the environment, 
can lead quite naturally to a quantity that grows with time, namely uncertainty itself. 
It is reasonable to presume that incomplete models of the evolution of a physical sys- 
tem will generate additional uncertainty in the reliability of the description of the 
system as they are evolved. If the velocities were all instantaneously reversed, in 
the hope that a previous sequence of events might be reversed, uncertainty would 
continue to grow within such a model. We shall need to quantify this vague notion 
of uncertainty, of course. Newton's laws on their own are time reversal symmetric, 
but intuition suggests that the injection and evolution of configurational uncertainty 
would break the symmetry. Entropy production might therefore be equivalent to the 
leakage of our confidence in the predictions of an incomplete model: an interpretation 
that ties in with prevalent ideas of entropy as a measure of information. 

Before we proceed further, we need to remind ourselves about the phenomenology 
of irreversible classical thermodynamic processes Q). A system possesses energy E 
and can receive additional incremental contributions in the form of heat dQ from a 
heat bath at temperature T, and work dW from an external mechanical device that 
might drag, squeeze or stretch the system. It helps perhaps to view dQ and dW 
roughly as increments in kinetic and in potential energy, respectively. We write the 
first law of thermodynamics (energy conservation) in the form dE = dQ + dW. The 
second law is then traditionally given as Clausius' inequality: 



where the integration symbol means that the system is taken around a cycle of heat and 
work transfers, starting and ending in thermal equilibrium with the same macroscopic 
system parameters, such as temperature and volume. The temperature of the heat 
bath might change with time, though by definition and in recognition of its presumed 
large size it always remains in thermal equilibrium, and so might the volume and 
shape imposed upon the system during the process. We can also write the second 
law for an incremental thermodynamic process as: 



where each term is an incremental entropy change, the system again starting and 
ending in equilibrium. The change in system entropy is denoted dS and the change 
in entropy of the heat bath, or surrounding medium, is defined as 




(1.1) 



dStot — dS + rfS'mod, 



(1.2) 
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such that dStot is the total entropy change of the two combined (the 'universe'). We 
see that equation l |l .1 1 corresponds to the condition <f> dStot > 0, since <f> dS = 0. A 
more powerful reading of the second law is that 



for any incremental segment of a thermodynamic process, as long as it starts and 
ends in equilibrium. An equivalent expression of the law would be to combine these 
statements to write dW — dE+TdS > 0, from which we conclude that the dissipative 
work (sometimes called irreversible work) in an isothermal process 



is always positive, where dF is a change in Helmholtz free energy. We also might 
write dS — dStot ~ dS mc d and regard dStot as a contribution to the change in entropy 
of a system that is not associated with a flow of entropy from the heat bath, the dQ /T 
term. For a thermally isolated system, where dQ — 0, we have dS = dStot and the 
second law then says that the system entropy increase is due to 'internal' generation; 
hence dStot is sometimes [7| denoted dSi. 

Boltzmann tried to explain what this ever-increasing quantity might represent at a 
microscopic level [6 |. He considered a thermally isolated gas of particles interacting 
through pairwise collisions within a framework of classical mechanics. The quantity 



where /(v, t)dv is the population of particles with a velocity in the range dv about 
v, can be shown to decrease with time, or remain constant if the population is in a 
Maxwell-Boltzmann distribution characteristic of thermal equilibrium. Boltzmann 
obtained this result by assuming that the collision rate between particles at velocities 
vi and V2 is proportional to the product of populations at those velocities, namely 
/(vi, t)/(v2, t). He proposed that H was proportional to the negative of system en- 
tropy and that his so-called //-theorem provides a sound microscopic and mechanical 
justification for the second law. Unfortunately, this does not hold up. As Loschmidt 
pointed out, Newton's laws of motion cannot lead to a quantity that always decreases 
with time: dH/dt < would be incompatible with the principle of time reversal 
symmetry that underlies the dynamics. The //-theorem does have a meaning, but it 
is statistical: the decrease in H is an expected, but not guaranteed result. Alternative- 
ly, it is a correct result for a dynamical system that does not adhere to time reversal 
symmetric equations of motion. The neglect of correlation between the velocities of 
colliding particles, both in the past and in the future, is where the model departs from 
Newtonian dynamics. 

The same difficulty emerges in another form when, following Gibbs, it is proposed 
that the entropy of a system might be viewed as a property of an ensemble of many 
systems, each sampled from a probability density P({x, v}), where {x, v} denotes 



dStot > 0, 
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dW d = dW-dF 



(1.5) 
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the positions and velocities of all the particles in a system. Gibbs wrote 



S G ibbs = -fcB J P({x,v})lnP({x,v}) J]dxdv, (1.7) 

where ks is Boltzmann's constant and the integration is over all phase space. The 
Gibbs representation of entropy is compatible with all of classical equilibrium ther- 
modynamics. But the probability density P for an isolated system should evolve in 
time according to Liouville's theorem, in such a way that S'Gibbs is a constant of the 
motion. How, then, can the entropy of an isolated system, such as the universe, in- 
crease? Either equation l |1.7} is only valid for equilibrium situations, something has 
been left out, or too much has been assumed. 

The resolution of this problem is that Gibbs' expression can represent thermody- 
namic entropy, but only if P is not taken to provide an exact representation of the 
state of the universe, or if you wish, of an ensemble of universes. At the very least, 
practicality requires us to separate the universe into a system about which we might 
know and care a great deal, and an environment with which the system interacts that 
is much less precisely monitored. This indeed is one of the central principles of 
thermodynamics. We are obliged by this incompleteness to represent the probability 
of environmental details in a so-called coarse-grained fashion, which has the effect 
that the probability density appearing in Gibbs' representation of the system entropy 
evolves not according to Liouville's equations, but to versions with additional terms 
that represent the effect of an uncertain environment upon an open system. This then 
allows Scibbs to change, the detailed nature of which will depend on exactly how the 
environmental forces are represented. 

For an isolated system however, an increase in Sctibbs W1 U emerge only if we are 
obliged to coarse-grain some aspect of the system itself. This line of development 
could be considered rather unsatisfactory, since it makes the entropy of an isolated 
system grain-size dependent, and alternatives may be imagined where the entropy 
of an isolated system is represented by something other than .S'Gibbs- The reader is 
directed to the literature [8 | for further consideration of this matter. However, in this 
chapter, we shall concern ourselves largely with entropy generation brought about 
by systems in contact with coarse-grained environments described using stochastic 
forces, and within such a framework the Gibbs' representation of system entropy will 
suffice. 

We shall discuss a stochastic representation of the additional terms in the system's 
dynamical equations in the next section, but it is important to note that a deterministic 
description of environmental effects is also possible, and it might perhaps be thought 
more natural. On the other hand, the development using stochastic environmental 
forces is in some ways easier to present. But it should be appreciated that some of the 
early work on fluctuation relations was developed using deterministic so-called ther- 
mostats (T][9j, and that this theme is represented briefly in section [L9] and elsewhere 
in this book. 



1.3 

Stochastic dynamics 



1.3.1 

Master equations 

We pursue the assertion that sense can be made of the second law, its realm of applica- 
bility and its failings, when Newton's laws are supplemented by the explicit inclusion 
of a developing configurational uncertainty. The deterministic rules of evolution of 
a system need to be replaced by rules for the evolution of the probability that the 
property should take a particular configuration. We must first discuss what we mean 
by probability. Traditionally it is the limiting frequency that an event might occur 
amongst a large number of trials. But there is also a view that probability represents 
a distillation, in numerical form, of the best judgement or belief about the state of 
a system: our information 1 10] . It is a tool for the evaluation of expectation values 
of system properties, representing what we expect to observe based on information 
about a system. Fortunately, the two interpretations lead to laws for the evolution of 
probability that are of similar form. 

So let us derive equations that describe the evolution of probability for a simple 
case. Consider a random walk in one dimension, where a step of variable size is 
taken at regular time intervals II 1II12|[T31 . We write the master equation describing 
such a stochastic process: 

oo 

"Pn+l( x m) = ^ Tn{x m — X m i\x m i )7>n(aw), (1.8) 
m' — — oo 

where V n (x m ) is the probability that the walker is at position x m at timestep n, and 
T n ( Ax | a;) is the transition probability for making a step of size A.t in timestep n given 
a starting position of x. The transition probability may be considered to represent the 
effect of the environment on the walker. We presume that Newtonian forces cause 
the move to be made, but we do not know enough about the environment to model 
the event any better than this. We have assumed the Markov property such that the 
transition probability does not depend on the previous history of the walker; only the 
position x prior to making the step. It is normalised such that 

oo 

T n (x m - X m ,\x m ,) = 1, (1.9) 

m— — oo 

since the total probability that any transition is made, starting from x m t, is unity. The 
probability that the walker is at position m at time n is a sum of probabilities of all 
possible previous histories that lead to this situation. In the Markov case, the master 
equation shows that these path probabilities are products of transition probabilities 
and the probability of an initial situation, a simple viewpoint that we shall exploit 
later. 



1.3.2 

Kramers-Moyal and Fokker-Planck equations 



The Kramers-Moyal and Fokker-Planck equations describe the evolution of proba- 
bility density functions, denoted P, which are continuous in space (K-M) and addi- 
tionally in time (F-P). We start with the Chapman-Kolmogorov equation, an integral 
form of the master equation for the evolution of a probability density function that is 
continuous in space: 



P{x,t + r)= J T(Ax\x - Ax,t)P(x - Ax,t)dAx. (1.10) 

We have swapped the discrete time label n for a parameter t. The quantity T(Ax|a;, t) 
describes a jump from x through distance Aa; in a period r starting from time t. 
Note that T now has dimensions of inverse length (it is really a Markovian transition 
probability density), and is normalised according to J T(Ax\x, t)dAx — 1. 

We can turn this integral equation into a differential equation by expanding the 
integrand in Ax to get 
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and define the Kramers-Moyal coefficients, proportional to moments of T: 

M n (x,t) = i I dAx(Ax) n T(Ax\x,t), (1.12) 

to obtain the (discrete time) Kramers-Moyal equation: 

1 (P(x, t + T) - P(x, t)) = ± ^(M n{x t )P M) 
r z — ' n! ox n 

n=l 

Sometimes the Kramers-Moyal equation is defined with a time derivative of P on the 
left hand side instead of a difference. 

Equation ( | 1 . 1 3[ > is rather intractable, due to the infinite number of higher derivatives 
on the right hand side. However, we might wish to confine attention to evolution 
in continuous time, and consider only stochastic processes which are continuous in 
space in this limit. This excludes processes which involve discontinuous jumps: the 
allowed step lengths must go to zero as the timestep goes to zero. In this limit, every 
coefficient vanishes except the first and second, consistent with the Pawula theorem. 
Furthermore, the difference on the left hand side of equation < | 1 - 1 3 1 ) becomes a time 
derivative and we end up with the Fokker-Planck equation (FPE): 

dP(x,t) d(MiQM)P(M)) i l d 2 (M 2 (x,t)P(x,t)) 

at dx ~ + 2 dz 2 ■ K ' 

We can define a probability current: 

J = M lM P M - ^ M ^ P ^\ (1.15) 



and view the FPE as a continuity equation for probability density: 

The FPE reduces to the familiar diffusion equation if we take M\ and M2 to be ze- 
ro and 2D, respectively. Note that it is probability that is diffusing, not a physical 
property like gas concentration. As an example, consider the limit of the symmetric 
Markov random walk in one dimension as timestep and spatial step go to zero: the 
so-called Wiener process. The probability density P(x, t) evolves according to 



dP{x,t) =D d 2 p(x,t) 

dt dx 



with an initial condition P(x, 0) = S(x). The statistical properties of the process are 
represented by the probability density that satisfies this equation: 

P(*,t)= ~ ^TTTo expf-^-V (1.18) 



representing the increase in positional uncertainty of the walker as time progresses. 
1.3.3 

Ornstein-Uhlenbeck process 

We now consider a very important stochastic process describing the evolution of the 
velocity of a particle. We shall approach this from a different point of view: a treat- 
ment of the dynamics where Newton's equations are supplemented by environmental 
forces, some of which are stochastic. It is proposed that the environment introduces 
a linear damping term together with random noise: 

v = -yu + b£(t), (1.19) 

where 7 is the friction coefficient, 6 is a constant, and £ has statistical properties 
(£(t)) = 0, where the brackets represent an expectation over the probability distribu- 
tion of the noise, and (f (t)£(t')) = S(t — t'), which states that the noise is sampled 
from a distribution with no autocorrelation in time. The singular variance of the noise 
might seem to present a problem but this can be accommodated. This is the Langevin 
equation. We can demonstrate that it is equivalent to a description based on a Fokker- 
Planck equation by evaluating the K-M coefficients, considering equation |TTT2j in 
the form 

M n (v,t) = - [ dAv(Av) n T(Av\v,t) = ~((v(t + T)-v(t)) n ), (1.20) 

T J T 

and in the continuum limit where r — > 0. This requires an equivalence between the 
average of (A«)™ over a transition probability density T, and the average over the 
statistics of the noise f. We integrate equation ( | 1 - 1 9[ > for small r to get 

rt+T rt+T r-t+T 

v(t + r) -v(t) = -7 J vdt + b J $(t')dt' « -7«(t)r + & J €(t')dt', 



(1.21) 



and according to the properties of the noise and in the limit r — > this gives (dv) = 
~jvt with dv — v(t + t) — v(t), such that M\(v) — (v) = —777. We also construct 
(v(t + t) — v(t)) 2 and using the appropriate statistical properties and the continuum 
limit, we get {(dv) 2 } = b 2 r and M2 = b 2 . We have therefore established that the 
FPE equivalent to the Langevin equation (JT7T9J is 

dP(v,t) _ d(~,vP(v,t)) b 2 d 2 p(v,t) 
dt dv + 2 a« 2 ' ( ' 

The stationary solution to this equation ought to be the Maxwell-Boltzmann velocity 
distribution P(v) oc exp (— mv 2 /2kgT^ of a particle of mass m in thermal equi- 
librium with a bath at temperature T, so b must be related to T and 7 in the form 
b 2 = 2kgT"/ /m, where k^ is Boltzmann's constant. This is a connection known as 
a fluctuation dissipation relation: b characterises the fluctuations and 7 the dissipa- 
tion or damping in the Langevin equation. Furthermore, it may be shown that the 



time-dependent solution to equation ( 1.22 1, with initial condition 5(v — vq) at time 

to, is 

P v[v,t\voM = ^/ 27rfcsT(1 _ e - 27(t - M) -P ( v - 2fcsT(l _ e -2 7 ( t - t0 ) ) 

(1.23) 

This is a gaussian with time-dependent mean and variance. The notation Pq V [■ ■ ■ ] is 
used to denote a transition probability density for this so-called Ornstein-Uhlenbeck 
process starting from initial value vq at initial time to, and ending at the final value v 
at time t. 

The same mathematics can be used to describe the motion of a particle in a har- 
monic potential <p(x) = kx 2 /2, in the limit that the frictional damping coefficient 7 
is very large. The Langevin equations that describe the dynamics are v — —jv — 
Kx/rn + b£(t) and x = v, which reduce in this so-called overdamped limit to 

x = — —x+ b ^(t), (1.24) 
7717 7 

which then has the same form as equation l |1.19[ , but for position instead of velocity. 
The transition probability |L23j, recast in terms of x, therefore can be employed. 

In summary, the evolution of a system interacting with a coarse-grained environ- 
ment can be modelled using a stochastic treatment that includes time-dependent ran- 
dom external forces. However, these really represent the effect of uncertainty in the 
initial conditions for the system and its environment: indefiniteness in some of those 
initial environmental conditions might only have an impact upon the system at a later 
time. For example, the uncertainty in the velocity of a particle in a gas increases as 
particles that were initially far away, and that were poorly specified at the initial time, 
have the opportunity to move closer and interact. The evolution equations are not 
time reversal symmetric since the principle of causality is assumed: the probability 



of a system configuration depends upon events that precede it in time, and not on 
events in the future. The evolving probability density can capture the growth in con- 
figurational uncertainty with time. We can now explore how growth of uncertainty 
in system configuration might be related to entropy production and the irreversibility 
of macroscopic processes. 

1.4 

Entropy generation and stochastic irreversibility 

1.4.1 

The reversibility of a stochastic trajectory 

The usual statement of the second law in thermodynamics is that it is impossible to 
observe the reverse of an entropy producing process. Let us immediately reject this 
version of the law and recognise that nothing is impossible. A ball might roll off a 
table and land at our feet. But there is never stillness at the microscopic level and, 
without breaking any law of mechanics, the molecular motion of the air, ground and 
ball might conspire to reverse their macroscopic motion, bringing the ball back to rest 
on the table. This is not ridiculous: it is an inevitable consequence of the time reversal 
symmetry of Newton's laws. All we need for this event to occur is to create the right 
initial conditions. Of course, that is where the problem lies: it is virtually impossible 
to engineer such a situation, but virtually impossible is not absolutely impossible. 

This of course highlights the point behind Loschmidt's paradox. If we were to 
time reverse the equations of motion of every atom that was involved in the motion 
of the ball at the end of such an event we would observe the reverse behaviour. Or 
rather more suggestively, we would observe both the forward and the reverse be- 
haviour with probability 1. This of course is such an overwhelmingly difficult task 
that one would never entertain the idea of its realisation. Indeed it is also not how 
one typically considers irreversibility in the real world, whether that be in the lab 
or through experience. What one might in principle be able to investigate is the 
explicit time reversal of just the motion of the particle(s) of interest to see whether 
the previous history can be reversed. Instead of reversing the motion of all the atoms 
of the ground, the air etc, we just attempt to roll the ball back towards the table at 
the same speed at which it landed at our feet. In this scenario we certainly would 
not expect the reverse behaviour. Now because the reverse motion is not inevitable 
we have somehow, for the system we are considering, identified (or perhaps con- 
structed) the concept of irreversibility albeit on a somewhat anthropic level: events 
do not easily run backwards. How have we evaded Loschmidt's paradox here? We 
failed to provide the initial conditions that would ensure reversibility: we left out the 
reversal of the motion of all the other atoms. If they act upon the system differently 
under time reversal then irreversibility is (virtually) inevitable. This is not so very 
profound, but what we have highlighted here is the one of the principle paradigms of 
thermodynamics, the separation of the system of interest and its environment, or for 
our example the ball and the rest of the surroundings. Given then that we expect such 



irreversible behaviour when we ignore the details of the environment in this way, 
we can ask what representation of that environment might be most suitable when 
establishing a measure of the irreversibility of the process? The answer to which is 
when the environment explicitly interacts with the system in such a way that time 
reversal is irrelevant. Whilst never strictly true, this can hold as a limiting case which 
can be represented in a model, allowing us to determine the extent to which the 
reversal of just the velocities of the system components can lead to a retracing of the 
previous sequence of events. Stochastic dynamics can provide an example of such a 
model. In the appropriate limits, we may consider the collective influence of all the 
atoms in the environment to act on the system in the same inherently unpredictable 
and dissipative way regardless of whether their coordinates are time reversed or not. 
In the Langevin equation this is achieved by ignoring a quite startling number of 
degrees of freedom associated with the environment, idealising their behaviour as 
noise along with a frictional force which slows the particle regardless of which way 
it is travelling. If we consider now the motion of our system of interest according to 
this Langevin scheme both its forward and reverse motion are no longer certain and 
we can attribute a probability to each path under the influence of the environmental 
effects. How might we measure irreversibility given these dynamics? It is no longer 
appropriate to consider whether upon time reversal the exact path is retraced since 
the paths are stochastic. Indeed in the continuous limit the probability of this hap- 
pening tends to zero. So we ask the question, what is the probability of observing 
some forward process compared to the probability of seeing that forward process 
undone? Or perhaps, to what extent has the introduction of stochastic behaviour 
violated Loschmidt's expectation? This section is largely devoted to the formulation 
of such a quantity. 

Intuitively we understand that we should be comparing the probability of observing 
some forward and reverse behaviour, but these ideas need to be made concrete. Let 
us proceed in a manner that allows us to make a more direct connection between 
irreversibility and our consideration of Loschmidt's paradox. First, let us imagine 
a system which evolves under some suitable stochastic dynamics. We specifically 
consider a realisation or trajectory that runs from time t = to t = r. Throughout 
this process we imagine that any number of system parameters may be subject to 
change. This could be, for example under suitable Langevin dynamics, the tempera- 
ture of the heat bath or perhaps the nature of a confining potential. The effect of these 
changes in the parameters is to alter the probabilistic behaviour of the system as time 
evolves. Following the literature we assume that any such change in these system 
parameters occurs according to some protocol X(t) which itself is a function of time. 
A particular realisation is not guaranteed to take place, since the system is stochastic, 
so consequently we associate with it a probability of occurring which will be en- 
tirely dependent on the exact trajectory taken, for example x(t), and the protocol \(t). 

We can readily compare different probabilities associated with different paths and 
protocols. To quantify an irreversibility in the sense of the breaking of Loschmidt's 
expectation however, we must consider one specific path and protocol. Recall now 



our definition of the paradox. In a deterministic system, a time reversal of all the 
variables at the end of a process of length r leads to the observation of the reverse 
behaviour with probability 1 over the same period r. It is the probability of the 
trajectory which corresponds to this reverse behaviour within a stochastic system 
that we must address. To do so let us consider what we mean by time reversal. A 
time reversal can be thought of as the operation of the time reversal operator, T 
on the system variables and distribution. Specifically for position x, momentum p 
and some protocol A we have Tx — x, Tp — —p and T\ — A. If we were to do 
this after time r for a set of Hamilton's equations of motion in which the protocol 
was time-independent, the trajectory would be the exact time reversed retracing of 
the forward trajectory. We shall call this trajectory the reversed trajectory and is 
phenomenologically the 'running backwards' of the forward behaviour. Similarly, 
if we were to consider a motion in a deterministic system that was subject to some 
protocol (controlling perhaps some external field), we would observe the reversed 
trajectory only if the original protocol were performed symmetrically backwards. 
This running of the protocol backwards we shall call the reversed protocol. We now 
are in a position to construct a measure of irreversibility in a stochastic system. We 
do so by comparing the probability of observing the forward trajectory under the 
forward protocol with the probability of observing the reversed trajectory under the 
reversed protocol following a time reversal at the end of the forward process. We 
literally attempt to undo the forward process and measure how likely that is. Since 
the quantities we have just defined here are crucial to this chapter we spend a mo- 
ment making their nature absolutely clear before we proceed. To reiterate we wish 
to consider a: 

• Reversed trajectory: 

Given a trajectory X(t) that runs from time t = to t = r, we define the reversed 
trajectory X(t) which runs forwards in time explicitly such that X (t) — TX(r —t). 
Examples are for position x(t) = x(r — t) and for momentum p(t) — —p(r — t). 

• Reversed protocol: 

The protocol A(t) behaves in the same way as the position variable x under time 
reversal and so we define the reversed protocol A(t) such that X(t) — A(r — t) 

Given these definitions we can construct the path probabilities we seek to compare. 
For notational clarity we label path probabilities that depend upon the forward proto- 
col A(t) with the superscript F to denote the forward process and probabilities which 
depend upon the reversed protocol X(t) with the superscript R to denote the reverse 
process. The probability of observing a given trajectory X, V F [X], has two com- 
ponents. First is the probability of the path given its starting point X(0) which we 
shall write as V F [X \t)\X '(0)] and second is the initial probability of being at the 
start of the path, which we write as P s tart {X(0)) since it concerns the distribution of 
variables at the start of the forward process. The probability of observing the forward 
path is then given as 

r F [x] = VstMx(o))r F [x(r)\x(o)]. (1.25) 

We can proceed along these lines; however it is often more intuitive to proceed if we 



imagine the path probability as being approximated by a sequence of jumps that occur 
at distinct times. Since continuous stochastic behaviour can be readily approximated 
by jump processes, but not the other way round, this simultaneously allows us to 
generalise any statements for a wider class of Markov processes. We shall assume 
for brevity that the jump processes occur in discrete time; however we note that the 
final quantity we shall arrive at would be identical if such a constraint were relaxed 
(we point the interested reader to [ 14], for example). By repeated application of the 
Markov property for such a system we can write 

V F [X] =Vst ax t(Xo)V(X 1 \Xo,\(.ti)) x P(X 2 \X 1 ,X(t 2 )) x ... 

...xV{X n \X n -i,\(t n )). (1.26) 

Here we consider a trajectory that is approximated by the jump sequence between 
n+ 1 points Xq ,X\ , . . . X n such that there are n distinct transitions which occur at dis- 
crete times t\, t 2 , ... t n , and where X = X(0)andX„ =X(t). 7?pQ|Xj_i, X(U)) 
is the probability of a jump from X;_i to Xj using the value of the protocol evaluated 
at time i, . 

Continuing with our description of irreversibility we construct the probability of 
the reversed trajectory under the reversed protocol. Approximating as a sequence of 
jumps as before we may write 

r R [x] = fr cnd (x(o))r R [x(r)\x(o)] 

= V^ T t(X(0))V R [X(T)\X(0)] 

= P B tert(*0)7>(*l|*0, A(ti)) x ... x V{X n \X n -x, X(t„)) (1.27) 

There are two key concepts here. The first, in accordance with our definition of irre- 
versibility, is that we attempt to 'undo' the motion from the end of the forward process 
and so the initial distribution is formed from the distribution to which Pstart evolves 
under X(t), such that for continuous probability density distributions we have 

P C nd(X(r)) = J dX P start (X(0))P F [X(T)\X(0)], (1.28) 

so named because it is the probability distribution at the end of the forward process. 
For our discrete model the equivalent is given by 

n-l 

^ond(^n) = ■ ■ ■ E II P(*i+ll*l.Afe+l))7W(*0). (1-29) 
X X n -! Z=0 

Secondly, to attempt to observe the reverse trajectory starting from X(t) we must 
perform a time reversal of our system to take advantage of the reversibility in Hamil- 
ton's equations. However, when we time reverse the variable X we are obliged to 
transform the distribution V en d as well, since the likelihood of starting the reverse 
trajectory with variable TX after we time reverse X is required to be the same as the 
likelihood of arriving at X before the time reversal. This transformed "P en( j, TV enA , is 



the initial distribution for the reverse process and is thus labelled Pgiarf Analogous- 
ly, evolution under A(t) takes the system distribution to P^ d - The forward process 
and its relation to the reverse process are illustrated for both coordinates x and v, 
which do and do not change sign following time reversal, respectively, in Figure [TT[ 
along with illustrations of the reversed trajectories and protocols. 

Let us now form our prototypical measure of the irreversibility of the path X, which 
for now we denote J: 



I[X] = In 



r F [x] 

P B [X] 



(1.30) 



There are some key points to notice about such a quantity. First, since X and X are 
simply related, 7 is a functional of the trajectory X and accordingly will take a range 
of values over all the possible 'realisations' of the dynamics: as such it will be char- 
acterised by a probability distribution. Further, there is nothing in its form which dis- 
allows negative values. Finally the quantity vanishes if the reversed trajectory occurs 
with the same probability as the forward trajectory under the relevant protocols: a 
process is deemed reversible if the forward process can be 'undone' with equal prob- 
ability. We can simplify this form since we know how the time reversed protocols 
and trajectories are related. Given the step sequence laid out for the approximation 
to a continuous trajectory we can transform X and t according to Xi = TX n _j and 
A(ij) = \{t n -i+i) giving 



P R [X] = V^ t {fX{r))V R [fX{0)\fX{r)] 

= P&rt(TX„)7>(fX n _i|fX n ,A(tn)) X 



xV(TX \TXi,\(ti)) 

(1.31) 



Pointing out that P R ta ,ATX n ) = TP end (TX n ) = V eTld {X n ) we thus have 



In 



P F [X] 



P R [X] 



V Pstart(X(0)) \ 

V r end (X(r)) J 



111 



V* [X(r)\X(0)\ 



hi 



V r [TX{Q)\TX(t 

TWtW-fx P(Xi\Xi- U \(ti)) 

To 



dW'S V(fx i -x\fx i ,\{t i ))_ 



(1.32) 



Let us explicitly consider this quantity for a specific model to understand its mean- 
ing in physical terms. Consider the continuous stochastic process described by the 
Langevin equation from section [T.3.3| where X — v and we have 



where { (t) is white noise. The equivalent Fokker- Planck equation is given by 

-)2 , 



(1.33) 



dP(v,t) _ d(-yvP{v,t)) k B T(t)j d z P(v,i) 
dt dv m dv 2 



(1.34) 



where P is a probability density. By insertion of probability densities and infinites- 
imal volumes into equation {L32} and cancelling the latter we observe that we may 
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Figure 1.1 An illustration of the definition of the forward and reverse processes. The 
forward process consists of an initial probability density, Pstart which evolves forward in 
time under the forward protocol A(t) over a period r, at the end of which the variable is 
distributed according to P en d- The reverse process consists of evolution from the 
distribution P$ alt , which is related to P cnd by a time reversal, under the reversed protocol 
\(t) over the same period r, at the end of which the system will be distributed according to 
some final distribution P^ d , which in general is not related to Pstart and does not explicitly 
feature in assessment of the irreversibility of the forward process. A particular realisation 
of the forward process is characterised by the forward trajectory X(t), illustrated here as 
being x(t) or v(t). To determine the irreversibility of this realisation, the reversed 
trajectories, x(t) or v(t), related by a time reversal, need to be considered as realisations 
in the reverse process. 



use probability densities to represent the quantity I[X] for this continuous behaviour 
without a loss of generality. To introduce a distinct forward and reverse process, let 
us allow the temperature to vary with a protocol X(t). We choose for simplicity a 
protocol which consists only of step changes such that 

T(X(ti)) = T 3 ti e [(j-l)At,jAt], (1.35) 

where j is an integer in the range 1 < j < N, such that NAt = r. Because the 
process is simply the combination of different Ornstein-Uhlenbeck processes each of 
which is characterised by defined solution equation ([L23J we can represent the path 
probability in a piecewise fashion. Consolidating with our notation, the continuous 
Langevin behaviour at some fixed temperature can be considered to be the limit, dt = 
(ti+i — U) 0, of the discrete jump process, so that 

h=jAt 

j™ II HvM-uKti)) = P^j[v(jAt)\v((j - l)At)]dv(jAt) = 

- 2An V /2 exp (_rn(vVM)-M-l)^**) a \ 

2wk B T j {l-e- 2 'y At )J P y 2k B Tj (l - e- 2 ~< At ) J U ' 

(1.36) 

The total conditional path probability density (with units equal to the inverse dimen- 
sionality of the path) over N of these step changes in temperature is then by applica- 
tion of the Markov property 

P F [v(r)\v(0)] = 

— / ~~ \V2 / m (v(jAt) - v((j -l)At)e-^ At f 



U ( 2irk B Tj(l - e- 2 7At) ) exp ^ 2k B Tj (1 - e-2-/ At ) 

(1.37) 

and since Tv — — v 



P r [- v (q)\-v(t)] = 

N , s 1/2 



n 



(1.38) 

Taking the logarithm of their ratio explicitly and abbreviating v(j At) = Vj yields 

P F [v(r)\v(0)] 



i(-v((j-l)At)+v(jAt)e-^ At Y 



lu 



P R [-v(0)\ -v(t)] 



k 



which is quite manifestly equal to the sum of negative changes of the kinetic energy 
of the particle scaled by k B and the environmental temperature to which the particle 



is exposed. Our model consists only of the particle and the environment and so each 
negative kinetic energy change of the particle, — AQ, must be associated with a pos- 
itive flow of heat AQ mo( j into the environment such that we define AQ mc( j = — AQ. 
For the Langevin equation the effect of the environment is idealised as a dissipative 
friction term and a fluctuating white noise characterised by a defined temperature 
which is entirely independent of the behaviour of the particle. This is the idealisation 
of a large equilibrium heat bath for which the exchanged heat is directly related to the 
entropy change of the bath through the relation AQ mc d = TAS. It may be argued 
that changing between N temperatures under such an idealisation is equivalent to 
exposing the particle to N separate equilibrium baths each experiencing an entropy 
change according to AQ mc( j j = TjASj. We consequently assert, for this particular 
model at least, that 



kg In 



P F [v(r)\v(0)] 
P R [-v(0)\ -v(t)] 



= V = V AS, = AS mcd (1.40) 

3 



where the entropy production in all N baths can be denoted as a total entropy pro- 
duction ASmcd that occurs in a generalised medium. 



Let us now examine the remaining part of our quantification of irreversibility which 



here is given in equation ( 1.32 1 by the logarithm of the ratio of P s tart("(0)) and 
Pcnd( v ( r ))- Given an arbitrary initial distribution one can write this as the change in 
the logarithm of the dynamical solution to P as given by the Fokker-Planck equation 
{L34}. Consequently we can write 

, ( f 8 tart(«(0))\ . P(v,0) , , . , 

111 { PcndHr)) ) = ln P^T) = - (lD P ^ T) " lnP{v ' 0)) ■ (L41) 

If we now characterise the mean entropy of our Langevin particle or 'system' with a 
Gibbs entropy which we allow to be time dependent such that 



<5 sys ) = 5 Gibbs = -k B J dv P(v,t)hxP(v,t) (1.42) 

one can make the conceptual leap that it is an individual value for the entropy of the 
system for a given v and time t that is being averaged in the above integral 
SWys = — kg In P(v,t). If we accept these assertions we find that our measure of 
irreversibility for any one individual trajectory is formed as 

k B I[X]=AS Bys + AS med . (1.43) 



Since our model consists only of the Langevin particle (the system) and a heat bath 
(the medium) we therefore regard this sum as the total entropy production associated 

2) Strictly P(v, t) is a probability density and so for equation jl.42| to be consistent with the entropy 
arising from the combinatoric arguments of statistical mechanics and dimensionally correct it might be 
argued we should be considering ln (P(v, t)dv). However, for relative changes this issue is irrelevant. 



with such a trajectory and make the assertion that our measure of irreversibility is 
identically the increase in the total entropy of the universe 



AStotpf] = AS'sys + AS'mod = Ub In 



P F [X] 
P R [X] 



(1.44) 



in this model at least. However, we have already stated that nothing prevents this 
quantity from taking negative values. If this is to be the total entropy production, 
how is this permitted given our knowledge of the second law of thermodynamics? 
In essence, describing the way in which a quantity that looks like the total entropy 
production can take both positive and negative values, but obeys well defined statis- 
tical requirements such that, for example, it is compatible with the second law, is the 
subject matter of the so-called fluctuation theorems or fluctuation relations. These 
relations are disarmingly simple, but allow us to make predictions far beyond those 
possible in classical thermodynamics. For this class of system in fact, they are so 
simple we can derive in a couple of lines a most fundamental relation and imme- 
diately reconcile the second law in terms of our irreversibility functional. Let us 
consider the average, with respect to all possible forward realisations, of the quantity 
exp (— AStot[X]/kB) which we write (exp (— AStot As)) an d where the angled 
brackets denote a weighted path integration. Performing the average yields 



-AS tot [X]/ki 



) = dX P b [X]e 



-AS tot [JC]/fe f 



P R [X] 



= / dX P F \X] , 
J 1 l P F [X 

= J dXP R [X] 



(1.45) 



where we assume the measures for the path integrals are related by dX = dX since 
the reverse trajectory is defined on the same space. Or perhaps more transparently, 
in the discrete approximation multiple summations over Xq, ■ ■ ■ X n yield the same 
result as summation over X n , . . . Xq. The expression above now trivially integrates 
to unity which allows us to write a so-called II 1 51 



Integral Fluctuation Theorem 



<* 



-AStotpq/fci 



> = i 



(1.46) 



This remarkably simple relation holds for all times, protocols and initial condition^] 
and implies that the possibility of negative total entropy change is obligatory. Further, 
if we make use of Jensen's inequality 



(exp (z)} > exp (z) 



(1.47) 



3) We do though assume that nowhere in the initial available configuration space do we have P s tart PO — 
0. This is a paraphrasing of the so-called ergodic consistency requirement found in deterministic systems 
|9| and insists that there must be a trajectory for every possible reversed trajectory and vice versa, so 
that the all possible paths, X(t), are included in the integral in the final line of equation 1 1 .451. 



we can directly infer 



(AStot) > 0. (1.48) 

Since this holds for any initial condition we may also state that the mean total en- 
tropy monotonically increases for any process. This statement, under the stochastic 
dynamics we consider, is the second law. It is a replacement of, or reinterpretation of 
equation < | 1 -4| >. The mean entropy production rate is always positive, but not necessar- 
ily in detail for individual realisations. The second law, when correctly understood, 
is statistical in nature and we have now obtained an expression which places a funda- 
mental bound on those statistics. 



1.5 

Entropy Production in the Overdamped Limit 

We have formulated a quantity which we assert to be the total entropy production, 
though it is for a very specific system and importantly has no ability to describe the 
application of work. To broaden the scope of application it is instructive to obtain a 
general expression like that obtained in equation ( |1.39} , but for a class of stochastic 
behaviour where we can formulate and verify the total entropy production without the 
need for an exact analytical result. This is straightforward for systems with detailed 
balance 1 16], however we can generalise further. The class of stochastic behaviour we 
shall consider will be the simple overdamped Langevin equation that we discussed in 
section [T. 3 . 3 1 involving a position variable described by 

i= m + (^T) 1/2 m (1 . 49) 



along with an equivalent Fokker-Planck equation 

dP(x,t) = 1 d(F(x)P(x,t)) | k B Td 2 P(x,t) 
dt nvy dx rwy dx 2 

The description includes a force term T(x) which allows us to model most simple 
thermodynamic processes including the application of work. We describe the force 
as a sum of two contributions which arise respectively from a potential <j>(x) and an 
external force f(x) which is applied directly to the particle, both of which we allow 
to vary in time through application of a protocol such that 

T(x, Ao(t), Ai(t)) = _M^oWl + f{X) Xl (t)). (1.51) 

The first step in characterising the entropy produced in the medium according to 
this description is to identify the main thermodynamic quantities including the heat 
exchanged with the bath. To do this we paraphrase Sekimoto and Seifert lfT7lll8lfT31 
and start from basic thermodynamics and the first law 



AE = AQ + AW 



(1.52) 



which must hold rigorously despite the stochastic nature of our model. To proceed, let 
us consider the change in each of these quantities in response to evolving our system 
by a small time dt and corresponding displacement dx. We can readily identify that 
the system energy for overdamped conditions is equal to the value of the conservative 
potential such that 

dE = dQ + dW = d{<t>{x,\ {t))). (1.53) 

However, at this point we reach a subtlety in the mathematics originating in the 
stochastic nature of x. Where normally we could describe the small change in <f> 
using the usual chain rule of calculus, when is a function of the stochastic variable 
x we must be more careful. The peculiarity is manifest in an ambiguity of expressing 
the multiplication of a continuous stochastic function by a stochastic increment. The 
product, which strictly should be regarded as a stochastic integral, is not uniquely 
defined because both function and increment cannot be assumed to behave smooth- 
ly on any timescale. The mathematical details 1 12] are not of our concern for this 
chapter and so we shall not rigorously discuss stochastic calculus, or go beyond the 
following steps of reasoning and assumption. First we assume that in order to work 
with thermodynamic quantities in the traditional sense, as in undergraduate physics, 
we require a small change to resemble that of normal calculus, and this requires, in all 
instances, multiplication to follow so-called Stratonovich rules. These rules, denoted 
in this chapter by the symbol o, are taken to mean evaluation of the preceding stochas- 
tic function at the mid-point of the following increment. Following this procedure we 
may write 

dE = d(<P{x(t),\ (t))) 

= d^(x(t),x (t))d\ Q (t) dt | d<j>{x(t),\o(t)) odx 

8Xq dt dx 

Next we can explicitly write down the work from basic mechanics as contributions 
from the change in potential and the operation of an external force: 

dW = Mg^Ml) + /(x(t)( Xl(t)) o dx . ( i.55) 

oaq dt 

Accordingly we directly have an expression for the heat transfer to the system in 
response to a small change dx 

dQ = ^M,Aoft)) Q dx _ f{x{t)t Ai(<)) Q dx 

= -J r (x(t),X (t),Xi(t))odx. (1.56) 
We may then integrate these small increments over a trajectory of duration r to find 



AE = / dE= d(^(a!(t),Ao(t))) = 0(x(T),Ao(T))-^(a!(O),Ao(O)) 
Jo JO 

= A</> (1.57) 



AW= f dW = f d^(tUo(t))dX (t) dt+ f f{x{t)Al{t))odx (L58) 
Jo Jo dX o at J 

and 

AQ = [ T dQ= [ T ^(^Mo^)) odx _ / T /( a; (t),A 1 (t))odx (1.59) 
Jo Jo " x Jo 

Let us now verify what we expect; that the ratio of conditional path probability den- 
sities we use in equation |L32j will be equal to the negative heat transferred to the 
system divided by the temperature of the environment. We no longer have a means 
for representing the transition probabilities in general and so we proceed using a so- 
called 'short time propagator' ifTTl 1 1 21 IT9l . which to first order in the time between 
transitions, dt, describes the probability of making a transition from Xi to ajj+i. We 
may then consider the analysis valid in the limit dt — > 0. The short time propa- 
gator can also be thought of as a short time Green's function; it is a solution to the 
Fokker- Planck equation subject to a delta function initial condition, valid as the prop- 
agation time is taken to zero. The basic form of the short time propagator is helpfully 
rather intuitive and most simply adopts a general gaussian form which reflects the 
fluctuating component of the force about the mean due to the gaussian white noise. 
Abbreviating T(x, Xo(t), Xi(t)) as T(x, t) we may write the propagator as 



P[xi + i,ti + dt\x l ,t i ] = 

2 



7717 

exp 



4irk B Tdt 



1 Xi + i - Xi dt 



4k B Tdt V ^ x 1 m 7 



(1.60) 



However, one must be very careful. For reasons similar to those discussed above, a 
propagator of this type is not uniquely defined, with a family of forms being available 
depending on the spatial position at which one chooses to evaluate the force, T, of 
which equation ( |1.60[ l is but one example [19|. In the same way we had to choose 
certain multiplication rules it is not enough to write T(x(€), t) on its own since x(t) 
hasn't been fully specified. This leaves a certain mathematical freedom in how to 
write the propagator and we must consider which is most appropriate. Of crucial 
importance is that all are correct in the limit dt — > (all lead to the correct solution 
of the Fokker-Planck equation) meaning our choice must rest solely on ensuring the 
correct representation of the entropy production. We can proceed heuristically: as 
we take time dt — > we steadily approach a representation of transitions as jump 
processes, from which we can proceed with confidence since jump processes are 
the more general description of stochastic phenomena. In this limit, therefore, we 
are obliged to faithfully represent the ratio that appears in equation ( |1.32[ l. In that 
description the forward and reverse jump probabilities have the same functional form 
and to emulate this we must evaluate the short time propagators at the same position 
x for both the forward and reverse transition. [^] Mathematically the most convenient 



4) For the reader aware of the subtleties of stochastic calculus we mention that for additive noise as con- 
sidered here this point is made largely for completeness: if one constructs the result using the relevant 



way of doing this is to evaluate all functions in the propagator midway between initial 
and final points. Evaluating the functions at the mid-point x such that 2x' = Xi+i + 
Xi and dx = ajj+i — Xi introduces a propagator of the form 



P[xi+i,h + dt\x i ,t i ] 



rwy 



exp 



rwy 



4nk B Tdt 

and similarly 

P[x l ,t i + dt\x l+1 ,ti 



Ak B Tdt 



dx _ r(x',t i ) dt 
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ld_ 
2dx' 



J~ (x , t j 
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dt 



(1.61) 



my 



Aixk B Tdt 



exp 



my 



Ak B Tdt 



my 



ld_ 
2dx' 



(x , 1 1 
my 



dt 

(1.62) 



The logarithm of their ratio, in the limit dt — > 0, simply reduces to 



lim In 

ctt->0 



P[xj+i,tj + dt\xj,ti} 
P[xi,ti + dt\x i+1 ,ti] 



In 



P(x 



Hk)) 



P(Xi\x i+ l, \(ti)) 

■F(g',A (^),Ai(t t )) 
k B T 

J"(a;(ii),Ao(ti),Ai(ti)) 



k B T 



o dx 



dQ 
k B T 



(1.63) 



where we get to the result by recognising that line two obeys our definition of 
Stratonovich multiplication rules since x' is the midpoint of dx and that line 3 con- 
tains the definition of an increment in the heat transfer from equation l |1.56[ . We can 
then construct the entropy production of the entire path by constructing the integral 
limit of the summation over contributions for each tj such that 



k B In 



P F [X(r)\X(0)] 



PR[X(t)\X(Q)] 



dQ- 



AQ 

T 



T 



(1.64) 



giving us the expected result noting that the identification of such a term from the 
ratio of path probabilities can readily be achieved in full phase space as well [20 1. 



1.6 

Entropy, Stationarity and Detailed Balance 

Let us consider the functional for the total entropy production once more, specifically 
with a view to understanding when we expect an entropy change. Specifically we aim 

stochastic calculus the ratio is independent of the choice. However to be a well defined quantity for 
cases involving multiplicative noise this issue becomes important. 



to identify two conceptually different situations where entropy production occurs. If 
we consider a system evolving without external driving, it will typically, for well 
defined system parameters, approach some stationary state. That stationary state is 
characterised by a time independent probability density, P st , such that 

^^=0. (1.65) 
ot 

Let us write down the entropy production for such a situation. Since the system is 
stationary we have P s tart = P C nd> but we also have a time- independent protocol 
meaning we need not consider distinct forward and reverse processes such that we 
write path probability densities P R — P F — P. In this situation the total entropy 
production for overdamped motion is given as 

' P st {x{Q))P[x(r)\x{Q)] 



AStotM = k B In 



P st (x(r))P[x(0)\x(T)] 



(1.66) 



We can then ask what in general are the properties required for entropy production, 
or indeed no entropy production in such a situation. Clearly there is no entropy pro- 
duction when the forward and reverse trajectories are equally likely and so we can 
write the condition for zero entropy production in the stationary state as 

P st (x(0))P[x(T)\x{0)] = P st (x(r))P[x{0)\x(r)} Vx{0),x(t). (1.67) 

Written in this form we emphasise that this is equivalent to the statement of detailed 
balance. Transitions are said to balance because the average number of all transi- 
tions to and from any given configuration a;(0) exactly cancel; this leads to a constant 
probability distribution and is the condition required for a stationary state. However 
to have no entropy production in the stationary state we require all transitions to 
balance in detail: we require the total number of transitions between every possible 
combination of two configurations x(0) and x(r) to cancel. This is also the condition 
required for zero probability current and for the system to be at thermal equilibrium 
where we understand the entropy of the universe to be maximised. 



We may then quite generally place any dynamical scheme into one of two broad 
categories. The first is where detailed balance (equation l |1.67[ l) holds and the 
stationary state is the thermal equilibrium |^] Under such dynamics systems left 
unperturbed will relax towards equilibrium where there is no observed preferen- 
tial forward or reverse behaviour, no observed thermodynamic arrow of time or 
irreversibility and therefore no entropy production. Thus all entropy production 
for these dynamics is the result of driving and subsequent relaxation to equilibrium 
or more generally as a consequence of the systems being out of their stationary states. 



5) One can build models which have stationary states that have zero entropy production where equilibrium 
is only local, but there is no value in distinguishing between the two situations or highlighting such cases 
here. 



The other category therefore is where detailed balance does not hold. In these situ- 
ations we expect entropy production even in the stationary state which by extension 
must have origins beyond that of driving out of and relaxation back to stationarity. 
So when can we expect detailed balance to be broken? We can first identify the 
situations where it does hold and for overdamped motion, the requirements are well 
defined. To have all transitions balancing in detail is to have zero probability current, 
J st (x,t) = 0, in the stationary state, where the current is related to the probability 
density according to 

dP st (x,t) dj st (x,t) 

dt ~ dx—=°- (L68) 

Utilising the form of the Fokker-Planck equation that corresponds to the dynamics 
we would thus require 



7T17 V " x J mr y " x 

(1.69) 

We can verify the consistency of such a condition by inserting the appropriate sta- 
tionary distribution 



P st (x, t) oc exp 



f x i 7717 f d(f>(x' ', A (t)) , i . A 

J dx ksfy — dx' — 



(1.70) 



which is clearly of a canonical form. How might one break this condition? We would 
require a non-vanishing current and this can be achieved when the contents of the ex- 
ponential in equation ( |1.70} are not integrable. In general this can be achieved by 
using an external force that is non-conservative. However in one dimension with nat- 
ural, that is reflecting, boundary conditions any force acts conservatively since the 
total distance between initial and final positions, and thus work done is always path 
independent. To enable such a non-conservative force one can implement periodic 
boundary conditions. This might be realised physically by considering motion on a 
ring since when a constant force acts on the particle the work done will depend on 
the number of times the particle traverses that ring. If the system relaxes to its sta- 
tionary state there will be a non-zero, but constant current that arises due to the non- 
conservative force driving the motion in one direction. In such a system with steady 
flow it is quite easy to understand that the transitions between two configurations will 
not cancel and thus detailed balance is not achieved. Allowing these dynamics to 
relax the system to its stationary state creates a simple example of a non- equilibrium 
steady state. Generally such states can be created by the placing of some constraint 
upon the system which stops it from reaching a thermal equilibrium. This results in 
a system which is perpetually attempting and failing to maximise the total entropy 
by equilibrating. By remaining out of equilibrium it constantly dissipates heat to 
the environment and is thus associated with a constant entropy generation. As such, 
a system with these dynamics gives rise to irreversibility beyond that arising from 



driving and relaxation and possesses an underlying breakage of time reversal sym- 
metry, leading to an associated entropy production, manifest in the lack of detailed 
balance. Detailed balance may be broken in many ways and the non- equilibrium con- 
straint that causes it may be, as we have seen, a non-conservative force, or it might 
be exposure to particle reservoirs with unequal chemical potentials or heat baths with 
unequal temperatures. The steady states of such systems in particular are of great in- 
terest in statistical physics, not only because of their qualitatively different behaviour, 
but also because they provide cases where analytical solution is feasible out of equi- 
librium. As we shall see later the distribution of entropy production in these states 
also obeys a particular powerful symmetry requirement. 



1.7 

A general fluctuation theorem 



So far we have examined a particular functional of a path and argued from a number 
of perspectives that it represents the total entropy production of the universe. We have 
also seen that it obeys a remarkably simple and powerful relation which guarantees 
its positivity on average. However, we can exploit the form of the entropy production 
further and derive a number of fluctuation theorems which explicitly relate distri- 
butions of general entropy-like quantities. They are numerous and the differences 
can appear rather subtle, however it is quite simple to derive a very general equality 
which we can rigorously and systematically adapt to different situations and arrive 
at these different relations. To do so let us once again consider the functional which 
represents the total entropy production 

>start(X(0))P F [X(r)|A-(0)r 



AStotpf] =k B ln 



P« (X(0))P«[X(r)|X(0)] 



(1.71) 



We are able to construct the probability distribution of this quantity for a particu- 
lar process. Mathematically, the distribution of entropy production over the forward 
process can be written as 



dX P st art(X(0))P i ' [X(t)\X{0)]S(A- AS tot [X]) 

(1.72) 



P* (AStot[X]=A) 



To proceed we follow Harris et al. 12 111 and consider a new functional, but one which 
is very similar to the total entropy production. We shall generally refer to it as R and 
it can be written 



R[X] = k B In 



b 

(X(0))P R [X(r)\X(0)} 



(1.73) 



Pstart(X(0))P F [X(r)|X(0)]_ 

Imagine that we evaluate this new quantity over the reverse trajectory, that is we con- 
sider R[X]. It will be given by 



R[X] = k B In 



b 

(X(0))P H [X(r)\X(0)} 
t (X(0))PF[X(r)\X(a)] 



-AStot[X] 



(1.74) 



which is explicitly the negative value of the functional that represents the total entropy 
production in the forward process. We can similarly construct a distribution for R[X] 
over the reverse process. This in turn would be given as 



P R (R[X] = A) = JdX P s R al . t (X(0))P R [X(r)\X(0)]8(A~R[X]). (1.75) 

We now seek to relate this distribution to that of the total entropy production over the 
forward process. To do so we consider the value the probability distribution takes for 
R[X] = — A. By the symmetry of the delta function we may write 

P R (R[X] = —A) = [ dX P s R alt (X(0))P R [X(r)\X(0)]5(A + R[X]). (1.76) 



We now utilise three substitutions. First, dX = dX denoting the equivalence of 
measure as the trajectories are defined on the same space. Next we use the definition 
of the entropy production functional to substitute 

P s R a rt(X(0))P R [X(r)\X(0)] =P start (X(0))P F {X(T)\X(0)]e~ AS ^™ kB 

(1.77) 

and finally the definition that R[X] = — AStotP^]- Performing the above substitu- 
tions we find 

P fl (i?[X] = -A)=|dXP s tart(X(0))P F [X(r)|X(0)]e"^r S 5(yl-AStot[X]) 

dXP start (X(0))P F [X(r)\X(0)}5(A-AStot[X}) 
= e _ *¥ P F (AS t ot[X]=A) (1.78) 

and yields 1211 

The Transient Fluctuation Theorem: 

P R (R[X] = -A) = P F (AS t ot[X] = A) (1.79) 

This is a fundamental relation and holds for all protocols and initial conditions and is 
of a form referred to in the literature as a finite time, transient or detailed fluctuation 
theorem depending on where you look. Additionally, if we integrate over all values 
of A on both sides we obtain the integral fluctuation theorem 

1 = ( e - A5tot / fcB ) (1.80) 

with its name now being self-explanatory. These two relations shall now form the 
basis of all relations we consider. However, upon returning to the transient fluctua- 
tion theorem, a valid question is what does the functional R[X] represent? In terms 
of traditional thermodynamic quantities there is scant physical interpretation. It is 



more helpful to consider it as a related functional of the path and to understand that 
in general it is not the entropy production of the reverse path in the reverse process. 
It is important now to look at why. To construct the entropy production under the 
reverse process, we need to consider a new functional which we shall call AS^ ot [X] 
that is defined in exactly the same way as for the forward process. We consider an 
initial distribution, this time P s tart which evolves to P R ld and compare the probabil- 
ity density for a trajectory starting from the initial distribution, this time under the 
reverse protocol A(t), with the probability density of a trajectory starting from the 



time reversed final distribution, TP^ d , so that 



^start 

(X(0))P R [X(r)\X(0)] 



^-AS to t[X]. (1.81) 



fpK d (X(0))pF[X(T)\X(0)] 
Crucially there is an inequality in equation < | 1 .8 1| ) in general because 

TPstart(X(0))) + P* d (X(r)) = J dX P s R alt (X(0))P R [X(r)\X(0)]. (1.82) 

This is manifest in the irreversibility of the dynamics of the systems we are looking 
at, as is illustrated in Figure [TTT] If the dynamics were reversible, as for Hamilton's 
equations and Liouville's theorem, then equation ( |1.82[ l would hold in equality. So, 
examining equations l |1.79} and ( |1.81[ l, if we wish to compare the distribution of 
entropy production in the reverse process with that for the forward process, we need 
to have R[X] = ASg, t [X] such that AS to t[X] = -AS^ ot [X]. This is achieved by 
having P s tart(^(0)) = fP* d (X(r)). When this condition is met we may write 

P R (AS? ot [X] = —A) = e-^P F (AS t ot[X] = A) (1.83) 

which now relates distributions of the same physical quantity, entropy change. If 
we assume that arguments of a probability distribution for the reverse protocol P R 
implicitly describe the quantity over the reverse process we may write it in its more 
common form 

P R (-AStot) = e- AStot/fcB P F (AS'tot) (1.84) 

This will hold when the protocol and initial distributions are chosen such that evo- 
lution under the forward process followed by the reverse process together with the 
appropriate time reversals brings the system back into the same initial statistical dis- 
tribution. This sounds somewhat challenging and indeed does not occur in any gen- 
erality, but there are two particularly pertinent situations where the above does hold 
and has particular relevance in a discussion of thermodynamic quantities. 

1.7.1 

Work Relations 



The first and most readily applicable example that obeys the condition P s tart (X (0) ) = 
TP R ld (X(T)) is that of changes between equilibrium states where one can trivially 



obtain the required condition by exploiting the fact that unperturbed, the dynam- 
ics will steadily bring the system into a stationary state which is invariant under 
time reversal. We start by defining the equilibrium distribution which represents 
the canonical ensemble where, as before, we consider the system energy for an 
overdamped system to be entirely described by the potential cj>(x, Xo(t)} such that 



P eq (x(t),X Q (t)) = 



Z(\ (t)) 



exp 



<j>(x(t),\ (t)) 
k B T 



(1.85) 



for t = and r, where Z is the partition function, uniquely defined by Ao(t), which 
can in general be related to the Helmholtz free energy through the relation 



F(X (t)) = -k B T In Z(X (t)). 



(1.86) 



To clarify, the corollary of these statements is to say that the directly applied force 
f(x(t), Ai(t)) does not feature in the system's HamiltoniarJ^] Let us now choose the 
initial and final distributions to be given by the respective equilibria defined by the 
protocol at the start and finish of the forward process and the same temperature 



Jstart(a;(0),Ao(0)) oc exp 
-Pend0c(7-),Ao(r)) oc exp 



JYAo(0))-#s(0),Aq(0)) " 
k B T 

F(\ (t))-<Kx(t),\q(t)) 
k B T 



(1.87) 



We are now in a position to construct the total entropy change for a given realisation 
of the dynamics between these two states. From the initial and final distributions we 
can immediately construct the system entropy change AS sys as 



AS, 



sys = k B In 

1 



f PstartQE(0),Ao(0)) 
\Pend(x(T),Xo(r)) 



k B In 



exp 


[>(Ao(0)) 


-4>(x(0), A (0))" 


k B T 


exp 




-0(z(t),A„(t))1 


k B T 



T (-F(A (r)) + F(A (0)) + 4>(x{t), A (r)) - cb(x(0), A (0))) 
Acf> — AF 



(1. 



The medium entropy change is as we defined previously and can be written 

AW — A(j> 



AQ 

T 



T 



(1.89) 



where AW is the work given earlier in equation l |1.58[ l, but we now emphasise that 
this term contains contributions due to changes in the potential and due to the external 



6) That is not to say it may not appear in some generalised Hamiltonian. For further insight into this issue 
we direct the interested reader to, for example |22 23], noting the approach here and elsewhere |24 
best resembles the extended relation used in 1221 . 
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force /. We thus further define two new quantities A Wo an d AWi such that AW 
AH'*; + AWi with 

AWo= rfl^oi^ 

Jo dx o at 

and 

AWi = / f(x(t),Xx(t))odx. (1.91) 



Wo and W\ are not defined in the same way with Wo being found more often in ther- 
modynamics and W\ being a familiar definition from mechanics: One may therefore 
refer to these definitions as thermodynamic and mechanical work respectively. The 
total entropy production in this case is simply given by 

AW - AF 

AStotN = T ■ d-92) 

Additionally, since we have established that P c ^ d (a;(r)) = TP s tart(a;(0)) we can also 
write 

fi AW- AF 

AStotM = y • (I- 93 ) 

1 .7.1 .1 The Crooks Work Relation and Jarzynski Equality 

The derivation of several relations follows now by imposing certain constraints on 
the process we consider. First let us imagine the situation where the external force 
f(x, Xi) — and so all work is performed conservatively through the potential such 
that AW = AW). To proceed we should clarify the form of the protocol that would 
take an equilibrium system to a new equilibrium such that its reversed counterpart 
would return it to the same initial distribution. This would consist of a waiting period, 
in principle of infinite duration, where the protocol is constant, followed by a period 
of driving where the protocol changes, followed by another infinitely long waiting 
period. Such a protocol is given and explained in Figure [L2| 
For such a process we write the total entropy production 

AS tot = ^AF (1.94) 

This changes its sign for the reverse trajectory and reverse protocol and so we may 
construct the appropriate fluctuation relation which is now simply read off equation 
(n79)as 



P F ((A W - AF)/T) = exp 



AW - AF 



P R (— (AWq — AF)/T) . (1.95) 



Since F and T are independent of the trajectory we can simplify and find [5 1 



The Crooks Work Relation: 



P F (AWq) _ 

P* (-AW ) 6XP 



AW - AF 



k B T 



(1.96) 



Ao(<) 



M') 



to td + T 2t„ + T 



Figure 1.2 A protocol A (t), of duration 2t + t, which evolves a system from one 
equilibrium to another defined so that the reversed protocol A (t) returns the system to the 
original equilibrium. There is a period of no driving of length t which corresponds to the 
relaxation for the reverse process, followed by a time r of driving, followed by another 
relaxation period of duration t a . As we take to to infinity we obtain a protocol which 
produces the condition P^ d (x(2t + r)) = fp start (i(0)) . We note here that 
/(*(t),Ai(t))=0. 




Rearranging and integrating over all AW on both sides, and taking the deterministic 
AF out of the path integral then yields an expression for the average over the forward 
process called Rll25l[T6l 

The Jarzynski Equality: 

(cxp (-AW /k B T)) = exp (-AF/fcflT) (1.97) 



The power of these statements is clarified in one very important conceptual point. 
In their formulation the relations are constructed using the values of entropy change 
for a process which, after starting in equilibrium, is isolated for a long time, driven 
and then left for a long time again to return to a stationary state. However this does 
not mean that these quantities have to be measured over the whole of such a process. 
Why is this the case? It is because the entropy production for the whole process 
can be written in terms of the mechanical work and free energy change which are 
delivered exclusively during the driving phase when the protocol Ao(i) is changing. 
Since the work and free energy change are independent of the intervals where the 
protocol is constant and because we had no constraint on Ao (t) during the driving 
phase we can therefore consider them to be valid for any protocol assuming the 
system is in equilibrium to start with. We can therefore state that the Crooks work re- 
lation and Jarzynski equality hold for all times for systems that start in equilibrium^*] 
Historically this has had one particularly important consequence: the results hold 
for driving, in principle, arbitrarily far from equilibrium. This is widely summed 
up as the ability to obtain equilibrium information from non-equilibrium averaging 
since, upon examining the form of the Jarzynski equality, we can compute the free 
energy difference by taking an average of the exponentiated work done in the course 
of some non- equilibrium process. Exploiting these facts let us clarify what these two 
relations mean explicitly and what the implications are in the real world. 



The Crooks Relation 
Statement: 

For any time r, the probability of observing trajectories which correspond to an 
application of A Wo work, starting from an equilibrium state defined by A(0), 
under dynamics described by A(i) in < t < r, is exponentially more likely 
in (A Wo — AF)/kgT than the probability of observing trajectories that cor- 
respond to an application of —AWq work from an equilibrium state defined by 
A(0), under dynamics described by A(t). 



Implication: 

Consider an isothermal gas in a piston in contact with a heat bath at equilibrium. 

7) Although we have only shown that this is the case for Langevin dynamics, it is important to note that 
these expressions can be obtained for other general descriptions of the dynamics. See for example [26]. 



Classically we know from the second law that if we compress the gas, perform- 
ing a given amount of work AWq on it, then after an equilibration period, we 
must expect the gas to perform an amount of work that is less than AWo when 
it is expanded (i.e. -AWo work performed on the gas). To get the same amount 
of work back out we need to perform the process quasistatically such that it is 
reversible. The Crooks relation however, tells us more. For the same example, 
we can state that if the dynamics of our system lead to some probability of 
performing AWo work, then the probability of extracting the same amount of 
work in the reverse process differs exponentially. Indeed they only have the same 
probability when the work performed is equal to the free energy difference, often 
called the reversible work. 

The Jarzynski Equality 

Statement: 

For any time r the average value, as defined by the mean over many realisations 
of the dynamics, of the exponential of the negative work divided by the tempera- 
ture arising from a defined change in protocol from Ao(0) to Ao(t) is identically 
equal to the exponential of the negative equilibrium free energy difference cor- 
responding to the same change in protocol, divided by the temperature. 

Implication: 

Consider once again the compression of a gas in a piston, but let us imagine that 
we wish to know the free energy change without knowledge of the equation of 
state. Classically, we might be able to measure the free energy change by at- 
tempting to perform the compression quasistatically; which of course can never 
be fully realised. However, the Jarzynski equality states that we can determine 
this free energy change exactly by repeatedly compressing the gas at any speed 
and taking an average of the exponentiated work that we perform over all these 
fast compressions. One must exercise caution however; the average taken is 
patently dominated by very negative values of work. These correspond to very 
negative excursions in entropy and are often extremely rare. One may find that 
the estimated free energy change is significantly altered following one additional 
realisation even if hundreds or perhaps thousands have already been averaged. 

These relations very concisely extend the classical definition of irreversibility in such 
isothermal systems. In classical thermodynamics we may identify the difference in 
free energy as the maximum amount of work we may extract from the system, or 
rather that to achieve a given free energy change we must perform at least as much 
work as that free energy change, that is 

AWo > AF (1.98) 

with the equality holding for a quasistatic 'reversible' process. But just as we saw that 
our entropy functional could take negative values there is nothing in the dynamics 
which prevents an outcome where the work is less than the free energy change. We 



understand now that the second law is statistical so more generally we must have 



(AWo) > AF. (1.99) 

The Jarzynski equality tells us more than this and replaces the inequality with an 
equality that it is valid for non-quasistatic processes where mechanical work is per- 
formed at a finite rate such that the system is driven away from thermal equilibrium 
and the process is irreversible. 

1.7.2 

Fluctuation relations for mechanical work 

Let us now consider a similar, but subtly different circumstance to that of the Jarzynski 
and Crooks relations. We consider a driving process that again starts in equilibrium, 
but this time keeps the protocol Xo(t) held fixed such that all work is performed by 
the externally applied force f(x(t), \i(t)) meaning that AW — AW±. Once again 
we seek a fluctuation relation by constructing an equilibrium to equilibrium process, 
though this time we insist that the system relaxes back to the same initial equilibrium 
distribution. We note that since f(x(t), X\(t)) may act non-conservatively, in order 
to allow relaxation back to equilibrium we would require that the external force be 
'turned off'. An example set of protocols is given in Figure [T3] for a simple external 
force f(x(t), Ai(t)) =A x (t). 
For such a process we find 

AW^ 

AS to t = -p- (1-100) 

since the free energy difference between the same equilibrium states vanishes. We 
have constructed a process such that the distribution at the end of the reverse process 
is (with time reversal) the same as the initial distribution of the forward process and 
so again we are permitted to read off a set of fluctuation relations |22, 23, 2711281 
which may collectively be referred to as 



Fluctuation relations for mechanical work: 
'AW!' 



V* (AWi) _ 
7>«(-AWi) ~ 6XP 



k B T 



(1.101) 



(exp (-AW 1 /k B T)) = 1 (1.102) 



For the same reasons as in the Jarzynski and Crooks relations they are valid for all 
times and thus hold as a non-equilibrium result. Taking in particular the integrat- 
ed relation and comparing with the Jarzynski equality in equation ( | 1 ,9T| > one may 
think there is an inconsistency. Both are valid for all times and arbitrary driving 
and concern the work done under the constraint that both start in equilibrium, yet 



Mt)Mt) 



M 
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Figure 1.3 An example protocol and reversed protocol that would construct the condition 
P^ d (x(2t + t)) = TP start (x(0)) when all work is performed through the external force 
f(x(t),\i(t)) = Xi(t) and to is taken to infinity. 



on first inspection they seem to be saying different things. But recall our distinction 
between the work AWq and AWi from equations |L90j and ( |1.91) ; there are two 
distinct ways to describe work on such a particle. If one performs work AWq one 
necessarily changes the form of the system energy whereas the application of work 
AW\ leaves the form of the system energy unchanged. The difference is manifest in 
the two different integrated relations because their derivations exploit the fact that the 
Hamiltonian, which represents the system energy, appears in initial and final distribu- 
tions. To clarify, as written the Jarzynski equality explicitly concerns driving where 
the application of any work also changes the Hamiltonian and thus the equilibrium 
state. On the other hand the relations for W\ concern work as the path integral of an 
external force such that the Hamiltonian remains unchanged for the entire process. 

Of course, there is nothing in the derivation of either of these relations that pre- 
cludes the possibility of both types of work to be performed at the same time and so 
using the same arguments we arrive at 

PR^W) = 6XP [{AW ~ AF )/ fcsT l (L1 ° 3) 
and 

_ A \V — A F 

(e "bt ) = i (1.104) 
again under the constraint that the system be initially prepared in equilibrium. 



1.7.3 

Fluctuation theorems for entropy production 



We have seen in section II. 7. II how relations between distributions of work can be 
derived from equation < | 1 .84| >, since work can be related to the entropy production 
during a suitable equilibrium to equilibrium process. We wish now to seek situations 
where we can explicitly construct relations that concern the distributions of the en- 
tropy produced for given forward and reverse processes that do not necessarily begin 
and end in equilibrium. In order to find situations where the value of the entropy 
production for the forward trajectory in the forward process is precisely the negative 
value of the entropy production for the reversed trajectory in the reverse process, we 
seek situations where the reverse protocol acts to return the distribution to the time 
reversed initial distribution for the forward process. For the overdamped motion we 
have been considering we would require 

Pstart (x(t)) = J dx P cnd (x(0))P R [as(T)|ar(0)] . (1.105) 

We seek a situation different from the previously considered equilibrium to equilibri- 
um process, occurring when the functional forms of the initial and final distributions 
are the same such that P s tart = P C nd- One can expect to see a symmetry between 



distributions of entropy production in such forward and reverse processes along the 
lines of equation ( |1 .84| >. 

However, we can specify further and find an even more direct symmetry if we 
insist that the evolution under the forward process is indistinguishable from that 
under the reverse process. Mathematically this means P(xi-\-i\xi, A(tj+i)) = 
P(x l+1 \xi, X(t i+1 )) or P R [x(r)\x(0)\ = P f [x(t)\x(0)]. Given these conditions 
evolution from the initial distribution will result in the final distribution and evo- 
lution under the reverse process from the final distribution will result in the initial 
distribution. If we consider in more detail the requirements for such a condition 
we understand there are two main ways in which this can be achieved. Given that 
the initial and final distributions are the same the first way is to require a constant 
protocol \(t). In this way the forward process is trivially the same as the reverse 
process. Alternatively we require the protocol to be time symmetric such that 
\(t) = A(r — £) = A(t). In both situations the forward and reverse processes are 
entirely indistinguishable. As such, by careful construction we can, in these specific 
circumstances, relate the probability of seeing a positive entropy production to that 
of a negative entropy production over the same forward process allowing us from 
equation ( |1.79) to write a 1151 
Detailed fluctuation theorem: 

P(AStot) = e ASM/kB P(-AStot) (1.106) 
Physically the two situations we have considered correspond to 

• fkart = fond, A(t) = const: 

To satisfy such criteria the system must be in a steady state, that is all intrinsic sys- 
tem properties (probability distribution, mean system entropy, mean system energy 
etc) must remain constant over the process. The simplest steady state is equilibrium 
which trivially has zero entropy production in detail for all trajectories. However, a 
non-equilibrium steady state can be achieved by breaking detailed balance through 
some constraint which prevents the equilibration as we saw in section |1.6| The 
mean entropy production rate of these states is constant, non-zero and, as we have 
now shown, there is an explicit exponential symmetry in the probability of positive 
and negative fluctuations. 

• Pstart = Pond = P, A(t) = A(t): 

This condition can be achieved in a system that is being periodically driven charac- 
terised by a time symmetric X(t). If from some starting point we allow the system 
to undergo an arbitrarily large number of periods of driving it will arrive at a so- 
called non-equilibrium oscillatory state such that P(x, t) = P(x, t + t p ) where t p 
is the period of oscillation. In this state we can expect the above relation to hold 
for integer multiples of period t p starting from a time such that \(t) = A(£). 



1.8 

Further results 



1.8.1 

Asymptotic fluctuation theorems 

In the class of system we have considered, the integral and detailed fluctuation theo- 
rems are guaranteed to hold. Indeed it has not escaped some authors' attention that 
the reason they do is fully explained in the very definition of the functionals they con- 
cern H29I . There is, however, a class of fluctuation theorems which does not have this 
property. These are known as asymptotic fluctuation theorems. Their derivation for 
Langevin and then general Markovian stochastic systems is due to Kurchan H30I and 
Lebowitz and Spohn [31 ] respectively, and superficially bear strong similarities with 
results obtained by Gallavotti and Cohen for chaotic deterministic systems Ii3l l32l . 
They generally concern systems that approach a steady state, and, for stochastic sys- 
tems, strictly in their definition concern a symmetry in the long time limit of the 
generating function of a quantity known as an action functional or flux 113 II . This 
quantity again concerns a trajectory that runs from xq through to x n , described using 
jump probabilities cr(a;j|a;j_i) and is given as 



W(t) = In 



cr(xi\xg) (T(x n \x n -i) 
a(xo\xi) ' ' ' a(x n -i\x n ) 



(1.107) 



The statement of such an asymptotic fluctuation theorem, which we shall not prove 
and only briefly address here, states that there exists a long time limit of the scaled 
cumulant generating function of W(t) such that 

e(s) = Urn --]n(exp[-sW(t)]) (1.108) 

and that this quantity possesses the symmetry 

e(s) = e(l-s). (1.109) 

From this somewhat technical definition we can derive a fluctuation theorem close- 
ly related to those which we have examined already. The existence of such a limit 
implies that the distribution function of the time averaged action functional W(t)/t, 
P(W(t) /t), follows a large deviation behaviour [33 ] such that, in the long time limit 
we have 

P{W{t)/t) ~ e ~ tf?(w/t) (1.110) 
where e is the Legendre transform of e defined as 



e(W/t) = max [e(s) - s(W /*)] 



(1.111) 



maximising over the conjugate variable s. Consequently using the symmetry relation 
of equation l |1.109[ we may write 



e(W/t) = max [e(s) - (1 - s){W/t)] 

S 

= max [e(s) + s(W/t)] - (W/t) 

S 

= e(-W/t) - (W/t). (1.112) 
Since we expect large deviation behaviour described by equation (JTTTTOj this implies 



P(W/t) ~ P{-W/t)e W (1.113) 
or equivalently 

P(W)~P(-W)e W (1.114) 

which is clearly analogous to the fluctuation theorems we have seen previously. Tak- 
ing a closer look at the action functional W we see that it is, for the systems we have 
been considering, a representation of the entropy produced in the medium or a mea- 
sure of the heat dissipated, up to a constant kg . Unlike the fluctuation theorems we 
considered earlier, this is not guaranteed for all systems. To get a basic understanding 
of this subtlety, we write the asymptotic fluctuation theorem for the medium entropy 
production for the continuous systems we have been considering in the form: 

P(AS ■ d ) ^ e AS meA /k B _ (1U5) 
P(-A6 mcd ) 

However, we know that when considering steady states the following fluctuation the- 
orem holds for all time 

P(AS tot ) AW**, (U16) 

P{ — A^totj 

Since AStot = AS^cd + ASsys we may understand that the asymptotic symmetry 
will exist when the system entropy change is negligible compared to the medium en- 
tropy change. In steady states we expect a continuous dissipation of heat and thus 
increase of medium entropy along with a change in system entropy that on average is 
zero. One might naively suggest that this guarantees the asymptotic symmetry since 
the medium entropy is unbounded and can grow indefinitely. This however, is not 
a strong enough condition since if the system configuration space is unbounded one 
cannot in general rule out large fluctuations to regions with arbitrarily low probability 
densities and therefore large changes in system entropy which in principle can persist 
on any timescale. What one requires to guarantee such a relation is the ability to ne- 
glect, in detail for all trajectories, the system entropy change compared with medium 
entropy change on long timescales. One can do so in general if we insist that the state 
space is bounded. This means that the system entropy has a well defined maximum 
and minimum value which can be assumed to be unimportant on long timescales and 



so the asymptotic symmetry necessarily follows. We note finally that systems with 
unbounded state space are ubiquitous and include simple harmonic oscillators 11341 
and so investigations of fluctuation theorems for such systems can and have lead to a 
wealth of non-trivial generalisations and extensions. 

1.8.2 

Generalisations and consideration of alternative dynamics 

What we hope the reader might appreciate following reading this chapter is the mal- 
leability of quantities which satisfy fluctuation relations. It is not difficult to produce 
quantities which obey relations similar to the fluctuation theorems (although it may 
be hard to show that they have any particular physical relevance) since the procedure 
simply relies on a transformation of a path integral utilising the definition of the en- 
tropy production itself. To clarify this point we consider some generalisations of the 
relations we have seen. Let us consider a new quantity which has the same form as 
the total entropy production 

~P F [X] 



G[X] =k B ln 



P[Y] 



(1.117) 



Here P F [X] is the same as before, yet we deliberately say very little about P[Y] 
other than it is a probability density of observing some path Y related to X defined 
on the same space as X. Let us compute the average of the negative exponential of 
this quantity 



dY P(Y(0))P[Y(t)\Y(0)} 



(1.118) 



As long as dX — dY and the unspecified initial distribution P(Y(0)) and transi- 
tion probability density P[F(t)| Y(0)] are normalised then any such quantity G[X] 
will obey an integral fluctuation theorem averaged over the forward process. Clearly 
there are as many relations as there are ways to define P[Y (r)| F(0)] and P(Y(0)) and 
most will mean very little at all [29 1. However there are several such relations in the 
literature obtained by an appropriate choice of P(Y(0)) and P[Y(r)|Y(0)] that say 
something meaningful including, for example, the Seifert end-point relations [35 1. 
We will very briefly allude to just two ways that this can be achieved by first noting 
that one may consider an alternative dynamics known as 'adjoint' dynamics, leading 
to conditional path probabilities written P a d[^( r )l^(0)], defined such that they gen- 
erate the same stationary distribution as the normal dynamics, but with a current of 
the opposite sign [36]. For the overdamped motion that we have been considering, 
where P F [X] = P Biaxi (x(0))P F [x(T)\x(0)], we may consider 



• Hatano-Sasa relation: 



By choosing 



P(Y(0))=J&rt(5(0)) (1.119) 
and 

P[y(r)|Y(0)] = p£[*(T)|z(0)] (1.120) 

we obtain the Hatano-Sasa relation 1 37 ] or integral fluctiation theorem for the 'non- 
adiabatic entropy production' [14, 38, 391 which concerns the so-called 'excess 
heat' transferred to the environment l401 such that 

(exp [-AQ ex /k B T - AS sys /k B ]) = 1. (1.121) 

The (negative) exponent here is best described as the entropy production associat- 
ed with movement to stationarity, which, phenomenologically includes transitions 
between non-equilibrium stationary states for which it was first derived. This use 
of the P^ d adjoint dynamics is frequently described as a reversal of both the pro- 
tocol and dynamics [36] to be contrasted with reversal of just the protocol for the 
integral fluctuation theorem. 

• Relation for the housekeeping heat: 

By choosing 

P(r(0))=Prtart(x(0)) (1.122) 
and 

P]Y(t)\Y(0)]=p£ 1 [x(t)\x(P)] (1.123) 

we obtain the Speck-Seifert integral relation |41] or integral fluctuation theorem 
for the 'adiabatic entropy production' 1 14. 38 39], which concerns the so-called 
'housekeeping heat' absorbed by the environment 11401 such that 

(exp[-AQ hk /fc B T]) = l (1.124) 

where AQ^ = AQ me( j — AQ CX [40], and where the negative exponent is best 
described as the entropy production or heat dissipation associated with the non- 
equilibrium steady state. Such a consideration might be called a reversal of the 
dynamics, but not the protocol. 

Both of these relations are relevant to the study of systems where detailed balance 
does not hold and amount to a division in the total entropy production, or irreversibil- 
ity, into the two types we considered in section [L6] namely movement towards sta- 
tionarity brought about by driving and relaxation, and the breaking of time reversal 
symmetry that arises specifically when detailed balance is absent. Consequently if 
detailed balance does hold then the exponent in equation {TTT24J is zero and equation 
< | 1 . 1 2 1 [ > reduces to the integral fluctuation theorem. 



1.9 

Fluctuation relations for reversible deterministic systems 

So far we have chosen to focus on systems that obey stochastic dynamics, where- 
by the interaction with the environment, and the explicit breakage of time reversal 
symmetry, is implemented through the presence of random forces in the equations 
of motion. However, there exists a framework for deriving fluctuation relations that 
is based on deterministic dynamical equations, whereby the environmental interac- 
tion is represented through specific non- linear terms |9] which supplement the usual 
terms in Newton's equations. These have the effect of constraining some aspect of 
the system, such as its kinetic energy, either to a chosen constant, or to a particular 
distribution as time progresses. Most importantly, they can be reversible, such that 
a trajectory driven by a specified protocol, and its time -reversed counterpart driven 
by a time-reversed protocol, are both solutions to the dynamics. In practice these so- 
called thermostatting terms which provide the non-linearity are taken to act solely on 
the boundaries (which can be made arbitrarily remote) in order for the system to be 
unaffected by the precise details of the input and removal of heat. This provides a par- 
allel framework within which the dynamics of an open system, and hence fluctuation 
relations, can be explored. Indeed it was through the consideration of determinis- 
tic, reversible dynamical systems that many of the seminal insights into fluctuation 
relations were first obtained [ 1 1. 

Given that there was a choice over the framework to employ, we opted to use 
stochastic dynamics to develop this pedagogical overview. This has some benefit 
in that the concept of entropy change can readily be attached to the idea of the growth 
of uncertainty in system evolution, identifying it explicitly with the intrinsic irre- 
versibility of the stochastic dynamics. Nevertheless, it is important to review the 
deterministic approach as well, and explore some of the additional insight that it pro- 
vides. 

The main outcome of seminal and ongoing studies by Evans, Searles and coworkers 
(UHMl is the identification of a system property that displays a tendency to grow with 
time under specified non-Hamiltonian reversible dynamics. The development of the 
if -theorem by Boltzmann was a similar attempt to identify such a quantity. However, 
we shall have to confront the fact that by their very nature, deterministic equations do 
not generate additional uncertainty as time progresses. The configuration of a system 
at a time t is precisely determined given the configuration at to. Even if the latter were 
specified only through a probability distribution, all future and past configurations 
associated with each starting configuration are fixed, and uncertainty is therefore not 
changed by the passage of time. Something other than the increase in uncertainty 
will have to emerge in a deterministic framework if it is to represent entropy change. 

Within such a framework, a system is described in terms of a probability density for 
its dynamical variables x, v, ■ ■ ■ , collectively denoted F. An initial probability density 
P(F, 0) evolves under the dynamics into a density P(T,t). Furthermore, the starting 
'point' of a trajectory Fq (namely (x(0),v(0) ■ ■ ■ ) is linked uniquely to a terminating 
point Ft (namely x(t), v(t), ■ ■ ■ ), passing through points F t r in between. It may then 



be shown 0]|2) that 



(-J A(r t ,)dt' 



p(r t ,t) = p(r ,o)exp - / A(r t ,)dt'), (1.125) 



where A(IV ) is known as the phase space contraction rate associated with configura- 
tion IV , which may be related specifically to the terms in the equations of motion that 
impose the thermal constraint. For a system without constraint, and hence thermally 
isolated, the phase space contraction rate is therefore zero everywhere, and the result- 
ing P(Tt,t) = P(ro, 0) is an expression of Liouville's theorem: the conservation of 
probability density along any trajectory followed by the system. 

For typically employed thermal constraints (denoted thermostats/ergostats, de- 
pending on their nature) it may be shown that the phase space contraction rate is 
related to the rate of heat transfer to the system from the implied environment. For 
the so-called Nose-Hoover thermostat at fixed target temperature T we are able to 
write / *A(r t /)dt' = AQ(r )/£;T, where AQ(r ) is the heat transferred to the 
system over the course of a trajectory of duration t starting from Vq. 

We now consider the dissipation function fl(r) defined through 

Q(T t ,)dt' =ln(^^^exp(- j\{F t 0dA\ , (1.126) 

where F^ is related to Ft by the reversal of all velocity coordinates. Assuming that 
the probability density at time zero is symmetric in velocities, such that P(Fq, 0) = 



P(Fq, 0) (which ensures that the right hand side of equation 1 1.126 1 vanishes when 
t — 0), we can write 

J Sl(F t ,)dt' = n t (F )t = In . d-127) 

defining a mean dissipation function Ot(ro) for the trajectory starting from Fq and 
of duration t. It is a quantity that will take a variety of values for a given protocol (the 
specified dynamics over the period in question), depending on To, and its distribution 
has particular properties, just as we found for the distributions of functionals such as 
AStot m equation (1.44) . For example, we have 

<ex P (-n t t)) = |dr P(r ,o)e xp (-n t (r )t) 

= J dFtP(Ft,t)exp(-n t t) 

dr t P(r t ,o) = i, (i.i28) 



where the averaging is over the various possibilities for To, or equivalently Ft, 
and where we have imposed a probability conservation condition droP(ro,0) = 
dFtP(Ft, t), implying that dFt is the region of phase space around Ft that contains 
all the end-points of trajectories starting within the region dFo around Fq. This 



result takes the same form as the integral fluctuation theorem obtained using stochas- 
tic dynamics, but now involves the mean dissipation function. In the deterministic 
dynamics literature, the result equation {T7T28} is known as a non-equilibrium par- 
tition identity. As a consequence, we can deduce that (&t) > 0, again a result that 
resembles several already encountered. 

Now let us consider a protocol that is time- symmetric about its midpoint, and for 
simplicity, consists of time variation in the form of the system Hamiltonian. The ther- 
mal constraint, as discussed above, is imposed through reversible non-Hamiltonian 
terms in the dynamics (let us say the Nose-Hoover scheme) and is explicitly time- 
independent and therefore isothermal. For such a case it is clear that a trajectory run- 
ning from Tq to Ft over the time period — ¥ t can be generated in a velocity-reversed 
form, and in reverse sequence, by evolving for the same period forwards in time under 
the same equations of motion, but starting from the velocity-reversed configuration 
at time t, namely F 1 . This evolution is precisely that which would be obtained by 
running a movie of the normal trajectory backwards. The velocity-reversed or time 
reversed counterpart to each phase space point T t i is visited, but in the opposite order, 
and the final configuration is Fq. The mean dissipation function for such a trajectory 
would be given by 

where AQ(T%) is the heat transfer for this time-reversed trajectory. The symmetry 
of the protocol, and the symmetry of the Hamiltonian under velocity reversal, allows 
us to conclude that the heat transfer associated with the trajectory starting from Fq is 
equal and opposite to that associated with starting point r*, and hence that the mean 
dissipation functions for the two trajectories must satisfy flt(ro) = — f2t(r*). We 
can then proceed to derive a specific case of the 

Evans-Searles Fluctuation Theorem (ESFT) 

associated with the distribution of values (it taken by the mean dissipation func- 
tion Q t (r ): 

P(fit) = f dF o P(F o ,0)S(Clt(F o )~Cl t ) = I dF t P(Ft,t)S(Cl t (Fo)-nt) 

= J dr t p(r t ,t)ex P (n t (r )t) ^'"j a (fi t (r ) - n t ) 

= exp(n t t) J dFtP(Ft,0)S(n t {Fo)-n t ) 

= exp(n t t) J dF t P(F t ,Q)8(-Q, t (Ft)-Q,t) 

= exp(n t t) I dF'iP(F'i,0)S(-nt(F'i)~Qt) 

= exp (Ott) P(-fit). ( U3 °) 



noting that the Jacobian for the transformation of the integration variables from Ft 
to is unity. Under the assumed conditions, therefore, we have obtained a relation 
that resembles (but historically preceded) the transient fluctuation theorem equation 
( |1.84[ i or detailed fluctuation theorem equation |TTT06j derived within the framework 
of stochastic dynamics. It only remains to make connections between the mean dissi- 
pation function and thermodynamic quantities to complete the parallel development, 
though it has been argued that the mean dissipation function itself is the more general 
measure of non-equilibrium behaviour [8]. 

If we assume that the initial distribution is one of canonical equilibrium, such that 
P(r , 0) oc exp (-H(Tq, 0)/fcT), where H(T , 0) is the system Hamiltonian at t = 
0, then we find from equation ( | 1 . 1 26[ > that 

fit(r )t = ~ (H(r t , o) - H(r , o)) - -L AQ(r ), (i.i3i) 

and if the Hamiltonian at time t takes the same functional form as it does at t = 0, 
then H(T t , 0) = H(T t ,t) and we get 

Q t (r )t = i(fl(r t ,i)-ff(r 0l o))-iAQ(r ) (i.m) 

= i (AE(r ) - AQ(r )) = ^AW(r ), (1.133) 

where AiJ(ro) is the change in system energy along a trajectory starting from Fq. 
Hence the mean dissipation function is proportional to the (here solely thermody- 
namic) work performed on the system as it follows the trajectory starting from Tq. 
We deduce that the expectation value of this work is positive, and that the probabil- 
ity distribution of work for a time-symmetric protocol, and starting from canonical 
equilibrium, satisfies the ESFT. 

Deterministic methods may be used to derive a variety of statistical results involv- 
ing the work performed on a system, including the Jarzynski equation and the Crooks 
relation. Non-conservative work may be included such that relations analogous to 
equation (l.lOlfr may be obtained. However, it seems that a parallel development of 
the statistics of AStot is not possible. The fundamental problem is revealed if we try 
to construct the deterministic counterpart to the total entropy production defined in 
equations l | 1 .44) and jl.64) : 



ASgt = AS sys + AS mcd = -fcln ^ J - ^ ■ (1.134) 

According to equation ( |1.125} this is identically zero. As might have been expected, 
uncertainty does not change under deterministic dynamics, and total entropy, in the 
form that we have chosen to define it, is constant. Nevertheless, the derivation of 
relationships involving the statistics of work performed and heat transferred, just al- 
luded to, corresponding to similar expressions obtained using the stochastic dynamics 
framework, indicate that the use of deterministic reversible dynamics is an equivalent 
procedure for describing the behaviour. Pedagogically it is perhaps best to focus on 
just one approach, but a wider appreciation of the field requires an awareness of both. 



1.10 

Examples of the fluctuation relations in action 



The development of theoretical results of the kind we have seen so far is all very well, 
but their meaning is best appreciated by considering examples, which we do in this 
section. We shall consider overdamped stochastic dynamics, such that the velocities 
are always in an equilibrium Maxwell-Boltzmann distribution and never enter into 
consideration for entropy production. And in the first two cases we shall focus on the 
harmonic oscillator, since we understand its properties well. The only drawback of 
the harmonic oscillator is that it is a rather special case and some of its properties are 
not general [29 42 1 though we deliberately avoid situations where the distributions 
produced are gaussian in these examples. In the third case we describe the simplest 
of non-equilibrium steady states and illustrate a detailed fluctuation theorem for the 
entropy, and its origin in the breaking of detailed balance. 

1.10.1 

Harmonic oscillator subject to a step change in spring constant 

Let us form the most simple model of the compression-expansion type processes 
that are ubiquitous within thermodynamics. We start by considering a 1-d classical 
harmonic oscillator subject to a Langevin heat bath. Such a system is governed by 
the overdamped equation of motion 

kx f 2k B T\ 1/2 ., . 

x = + — 5- £(t), (1.135) 

rrvy \ m^y J 

where k is the spring constant. The corresponding Fokker-Planck equation is 

dP{x,t) = 1 d( K xP(x,t)) | k B Td 2 P(x,t) 
dt rwy dx 7717 dx 1 

We consider a simple work process whereby, starting from equilibrium at temperature 
T, we instigate an instantaneous step change in spring constant from kq to ki at t — 
with the system in contact with the thermal bath at all times. This has the effect of 
compressing or expanding the distribution. We are then interested in the statistics of 
the entropy change associated with the process. Starting from equations ( |1.44} and 
l |1.64) for our definition of the entropy production we may write 

A Stot = A ^ +fcB ln(^g)), (1.137) 

utilising notation x\ — x(t) and xq = x(0) and <j){x) = kx 2 /2. We also have 

AW= \{ki-k )xI, (1.138) 
and 

A<?f> = ^n\x\ - ^ko^o, (1.139) 



and so can write 

AStot =-|(«?- 4) +k B ln( ^Epl) , U 40, 

Employing an initial canonical distribution 

\ 1/2 / 2 \ 



the distribution at the end of the process will be given by 

/oo 
dx Poij[ x ll :r o]^start(^o), 
-oo 



(1.142) 



This is the Ornstein-Uhlenbeck process and so has transition probability density Pq V 
given by analogy to equation (1.23) . Hence we may write 



l/ 2 / 



-Pcnd(zi) = / dx Q | j± j—^ | exp 



«i(a;i-xoe -7 J 



\ 1/2 / 2 \ 

2nk B T P l 2fc B ri 



^V /2 ex P f-fl|V (1.143) 



\2-Kk B T J ^\ 2k B T)' 



where 



m = = T\ , (i.i44) 

such that P on d is always gaussian. The coefficient ft(t) evolves monotonically from 
kq at t — to ki as t — > oo. Substituting this into equation ( |1.140} allows us to write 



AC I + \ Kl ( 2 2\ . k B ( K \ K X% n(t)x\ 

A5tot(xi,x ,t) = [an - xoj + - In (j^yj - ^ + (1.145) 

for the entropy production associated with a trajectory that begins at xo and ends at 
x\ at time t, and is not specified in between. We can average this over the probability 
distribution for such a trajectory to get 



(AStot) = / dx dxiPo\j[xi\xo]P staj rt(xo)AStot(xi,xo,t) 



making full use of the separation of AStot into quadratic terms, and the gaussian 
character of the distributions. At t — (AStot) i s zero, and as t — > oo we find 



lim <AS tot ) = - - 1 - In ^ , (1.147) 

which is positive since In 2 < jz — 1 for all z. Furthermore, 

d(AStot) _ k B dk 



dt 2k 2 dt 



(ni-ii), (1.148) 



and it is clear that this is positive at all times during the evolution, irrespective of 
the values of k\ and kq. If k\ > kq then k increases with time, whilst remaining 
less than k\, and all factors on the right hand side of equation ( | 1 . 148[ > are positive. 
If «i < kq then k decreases but always remains greater than k\ and the mean total 
entropy production is still positive as the relaxation process proceeds. 

The work done on the system is simply the input of potential energy associated 
with the shift in spring constant: 

&W{x l ,x Q ,t) = ) 2 {K 1 -K Q )xl, (1.149) 
and so the mean work performed up to any time t > is 

(AW) = ^(^-l), (1.150) 



2 \K 

which is greater than AF = (fesT/2) In (ki/kq). The mean dissipative work is 

{AW d ) = (AW) - AF = ^ ( ^ - 1 - In ( ^) ), (1.151) 

and this equals the mean entropy generated as t — > 00 derived in equation l |1.147[ , 
which is to be expected since the system started in equilibrium. More specifically, let 
us verify the Jarzynski equality: 

(cxp (—AW/kgT)} = I dxoPstartOo) exp (- («i - « ) xt/2k B T^ 

= (koM) 1/2 = exp (-AF/k B T) , (1.152) 



as required. 



«0 



Now we demonstrate that the integral fluctuation relation is satisfied. We consider 

<«p(-aw« - («p (sgf (J - J) - i m (a ) + ^| - ^L)) 

= I da;idx Poij[ :r l! x o]-Pstart(a;o) 
dx\dxQ ^ 

\ 1/2 

2nk B Tj 6XP V 2k B T J ""^ \2k B T V 1 ~ u / ' 2k B T 2k B T 
= 1, (1.153) 

which is a tedious integration, but the result is inevitable. 

Further we can directly affirm the Crooks relation for this process. The work over 
the forward process is given by equation l |1.149[ l and so, choosing k\ > kq, we can 
relate its distribution according to 

P F (AW) = P cq (x )- d ' J " 




dAW 

/v„ \ 1/2 ( «o*g\ 1 / 2AW \~ 1/2 



■>,rl exp [-W^ ^- H(AW) 
— ( '■ " AW- ^ exp f ^ W/ ( A I V ) 



y/Ank B T \K\ — kq J \ ki-ko k B T 



(1.154) 



where H(AW) is the Heaviside step function. Let us consider the appropriate reverse 
process. Starting in equilibrium defined by k\, where again to form the reversed 
protocol we must have k\ > kq, the work is 

AW=i(/so-Ki)z? (1.155) 
and so we can relate its distribution according to 

P R (AW) = P cq (x 1 )- dX1 



dAW 

m \ 1/2 / kix\ \ 1 / 2AW x_1/2 



2nk B T J 6XP ^ 2k B T) k - m \k - «i ) ^ 
^Ank B T \k, '^iJ V KQ-Kik B Tj 

(1.156) 



so that 



P (— AW) = (-AW)- 1 ^ exp -^i— ~ H(Aff) 
1 ( *i \ 1/2 (AW) -i/2 exp / AW\ lJiAU ! 



^4nksT \ K± — fco / \ «1 — ko kgT 

(1.157) 

Taking the ratio of these two distributions ( | 1 . 1 54| > and l |1.157} gives 
P F (AW) fa / ko-kiAW\ 

p fl (-AW) ~~ V m exp V ki - ko y 



exp 



AW - ^ In £± 
k B T 



I AW — AF ■ 

exp — — (1.158) 



which is the Crooks work relation as required. 
1.10.2 

Smoothly squeezed harmonic oscillator 

Now let us consider a process where work is performed isothermally on a particle, 
but this time by a continuous variation of the spring constant. We have 

AW= r^fM^, (i.i59) 

Jo dx dt 
where A(t) = n(t) and <f>(x(t), «(t)) = |/t(t)x|, where a;t = a:(i), such that 

AW= / -k{t)x 2 t dt. (1.160) 
./o 2 

Similarly the change in system energy will be given simply by 

[ T dd>(x(t),\(t)) u 1 . , 2 1 2 
= Jo d * = 2 ^ ~ 2 ' 

Accordingly we can once again describe the entropy production as 

AStot = — — / kxfdt — «(t):e2h — — Knxn + kn In | jjjjjgj^Oj ] (1.162) 

2T y * 2T w T 2T u u B \P C nd(x T ) J 

For convenience, we assume the initial state to be in canonical equilibrium. The 
evolving distribution P satisfies the appropriate Fokker-Planck equation: 

OP = K (t) d{xP) | k B Td 2 P 
dt ra'y dx my dx 2 



Since P is initially canonical, it retains its gaussian form and can be written 

P end{xT) = P( XT ,r)= (^) 1/2 ex P (-|g^), d.164) 
where k(t) has its own equation of motion according to 

^ = — — k(k — k) , (1.165) 

dt m-y 

with initial condition k(0) = no. We can solve for k: write z = k~ x such that 

5 = — (1-kz). (1.166) 
This has integrating factor solution 

z(r)expf — / = z(0)+ / cxpf— [ K(t')dt'\ —dt, (1.167) 

V m 7./o / Jo \ mr y Jo /™7 



or equivalently 

1 1 

k(r) ~ k(0) 



(1.168) 

Returning to the entropy production, we now write 

(1.169) 

and we also have 

AW = ( \kx\dt. (1.170) 
Jo 2 

We can investigate the statistics of these quantities: 

(AW) = [ T \k{x 2 t )dt = f h^ldt, (1.171) 
Jo z Jo z K 

and from AW^ = AW — AF, the rate of performance of dissipative work is 

d{AW d ) = kk B T dF( K (t)) = kk B T kk B T = k B T . (1 1\ 
dt 2ft dt 2k 2k 2 V« k/ 

Whilst the positivity of (AWd) is assured for this process, as a consequence of the 
Jarzynski equation and the initial equilibrium condition, the rate of change can be 
both positive and negative, according to this result. 



The expectation value for total entropy production in equation l |1.169[ l, on the other 
hand, is 



/ac \ 1 f 7 1 ■ k B T 1 i \ k B T 1 , „ k B ( k \ k B T k 
<A5 tot ) = -J - K _d t __ K ( T )_ + _fc B r+-In^ j- — + 



(1.173) 



and the rate of change of this quantity is 

d(AStot) _ kk B k B ( k k -\ _ k B k 

dt 2k ~T \k k? K ) ~Yk 

k B i (/t — k) 



2 k? 
k B (a — k)" 2 
m~f k 



(1.174) 



The monotonic increase in entropy with time is explicit. The mean dissipative work 
and entropy production for a process of this kind starting in equilibrium are illus- 
trated in Figure [L4| where the protocol changes over a driving period followed by a 
subsequent period of equilibration. Notice particularly that the mean entropy pro- 
duction never exceeds the mean dissipative work, which is delivered instantaneously, 
and that both take the same value as t — > oo giving insight into the operation of the 
Jarzynski equality as discussed in section [T.7.1| 

It is of more interest however, to verify that detailed fluctuation relations hold. 
Analytic demonstration based upon equation l |1.169[ l and the probability density for 
a particular trajectory throughout the entire period is challenging, but a numerical 
approach based upon generating sample trajectories is feasible particularly since the 
entire distribution can always be characterised with the known quantity k(t). As 
such we may consider the same protocol, «(f) = sin 2 (7rf) + 1, wait until the system 
has reached a non-equilibrium oscillatory steady state as described in section[T.7.3| 



characterised here by an oscillatory k(t) as seen in Figure 1.4 and measure the en- 
tropy production over a time period across which /t(i) is symmetric. The distribution 
in total entropy production over such a period and the symmetry it possesses are 
illustrated in Figure [T3] 

1.10.3 

A simple non-equilibrium steady state 

Let us construct a very simple non-equilibrium steady state. We consider an over- 
damped Brownian motion on a ring driven in one direction by a non-conservative 
force. We may for sake of argument assume a constant potential cj>(x) — c such that 
the equation of motion is simply 

f + ( 2k ^) 1/2 m. (i.i75) 
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Figure 1.4 An illustration of a the mean behaviour of the dissipative work and entropy 
production for an oscillatory compression and expansion process starting in equilibrium. 
The mean dissipative work increases, but not monotonically, and is delivered instantly 
such that there is no change when the protocol stops changing. The mean entropy 
production however, continues to increase monotonically until it reaches the mean 
dissipative work after the protocol has stopped changing. The evolution of the protocol, 
n(t) = sin 2 (7rt) + 1, and the characterisation of the distribution, k(t), are shown 
inset. Units are fcs = T = m = 7 = 1. 




Figure 1.5 An illustration of a detailed fluctuation theorem arising for an oscillatory 
non-equilibrium steady state, as described in section QTT3] created by compressing and 
expanding a particle in an oscillator by using protocol X(t) = n(t) = sin 2 (7rf) + 1. The 
total entropy production must be measured over a time period during which the protocol is 
symmetric and the distribution is deemed to be oscillatory. Such a time period exists 
between t = 3 and t = 4 as shown inset in Figure [T~4| Units are fes = T = m = 7 = 1. 



This is just the Wiener process from equation < | 1 - 1 8| ) centred on a mean proportion- 
al to the external force multiplied by the time so the probability density of a given 
displacement is defined by 



P[x(r)\x(Q)] 
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ATvkgTr 



exp 



m 7 ( Ax ~ 4^ T ) 
Ak B Tr 



(1.176) 



where the lack of a superscript on P recognises the constancy of the protocol, and 
noting that Ax — x(t) — x(0) may extend an arbitrary number of times around the 
ring. Additionally by utilising the symmetry of the system we can trivially state that 
the stationary distribution is given by 



P s \x) = L- 1 



(1.177) 



where L is the circumference of the ring. Let us consider the nature of the dynam- 
ics in this steady state. Considering that we are in the steady state we know that the 
transitions must balance in total; however let us consider the transitions between in- 
dividual configurations: comparing the probabilities of transitions we immediately 
see that 



P St (x(0))P[x(T)\x(0)] / P St (x(T))P[x{0)\x(T)}. 



(1.178) 



Detailed balance explicitly does not hold. For this system not only can there be en- 
tropy production due to driving such as is the case with expansion and compression 
processes, but there is a continuous probability current in the steady state, in the di- 
rection of the force, which dissipates heat into the thermal bath. We have previously 
stated in section [L73] that the distribution of the entropy production in such steady 
states obeys a detailed fluctuation theorem for all times. Let's verify that this is the 
case. The entropy production is rather simple and is given by 



AS t , 



kg In 



P st (x(0))P[x(r)\x(0)} 
P st {x(T))P[x(0)\x{T)] 
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(1.179) 



This provides an example where the entropy production is highly intuitive. Taking 
/ > 0, if the particle moves with the probability current, Ax > 0, it is doing the 
expected behaviour and thus is following an entropy generating trajectory. If how- 
ever, the particle moves against the current, Aa; < 0, it is behaving unexpectedly 
and as such is performing a trajectory that destroys entropy. It follows that since an 
observation of the particle flowing with a current is more likely than an observation 
of the opposite, then on average the entropy production is positive. 



Since the system is in a steady state we expect a detailed fluctuation theorem. The 
transformation of the probability distribution is trivial and we have simply 

and we can verify a detailed fluctuation theorem which holds for all time. We can 
probe further though. Whilst we might conceive of some fluctuations against a steady 
flow for a small particle we would be quite surprised to see such deviations as we 
approached a macroscopically sized object. Despite the model's limitations let us 
consider an approach to macroscopic behaviour whilst maintaining constant the ratio 
/ jm such that the mean particle velocity is unchanged. Both the mean and vari- 
ance of the distribution of entropy production increase in proportion. On the scale of 
the mean, the distribution of entropy change increasingly looks like a narrower and 
narrower gaussian until it inevitably, for a macroscopic object, approaches a delta 
function where we recover the classical thermodynamic limit. 

1.11 

Final remarks 

The aim of this chapter was to explore the origin, application and limitations of fluc- 
tuation relations. We have done this within a framework of stochastic dynamics with 
white noise and often employing the overdamped limit in example cases where the 
derivations are easier: it is in the analysis of explicit examples where understand- 
ing is often to be found. Nevertheless, the results can be extended to other more 
complicated stochastic systems, though the details will need to be sought elsewhere. 
The fluctuation relations can also be derived within a framework of deterministic, 
reversible dynamics, which we have discussed briefly in section [L9| It is interest- 
ing to note that within that framework, irreversibility finds its origins in non-linear 
terms which provide a contraction of phase space, in contrast to the more direct irre- 
versibility found in stochastic descriptions. Both approaches, however, are attempts 
to represent a dissipative environment that imposes a thermal constraint. 

The fluctuation relations concern the statistics of quantities associated with thermo- 
dynamic processes, in particular the mechanical work done upon, or the heat trans- 
ferred to a system in contact with a heat bath. In the thermodynamic limit, the statis- 
tics are simple: there are negligible deviations from the mean, and work and heat 
transfers appear to be deterministic and the second law requires entropy change to be 
non-negative. But for finite size systems, there are fluctuations, and the statistics of 
these will satisfy one or more fluctuation relation. These can be very specific require- 
ments, for example relating the probability of a fluctuation with positive dissipative 
work to the probability of a fluctuation with negative dissipative work in the reversed 
process. Or the outcome can take the form of an inequality that demonstrates that the 
mean dissipative work over all possible realisations of the process is positive. 
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The core concept in the analysis, within the framework of stochastic dynamics at 
least, is entropy production. This no longer need be a mysterious concept: it is a 
natural measure of the departure from dynamical reversibility, the loosening of the 
hold of Loschmidt's reversibility expectation, when system interactions with a coarse- 
grained environment are taken into account. Entropy production emerges in stochas- 
tic models where there is uncertainty in initial specification. Intuitively, uncertainty 
in configuration in such a situation will grow with time, and mean entropy produc- 
tion is this concept commodified. And it turns out that entropy production can also 
be related, in certain circumstances, to heat and work transfers, allowing the growth 
of uncertainty to be monitored in terms of thermodynamic process variables. More- 
over, although it is expected to be positive, entropy change can be negative, and the 
probability of such an excursion, possibly observed by a measurement of work done 
or heat transferred, might be described by a fluctuation relation. In the thermody- 
namic limit the entropy production appears to behave deterministically and to violate 
time reversal symmetry, and only then does the second law acquire its unbreakable 
status. But for small systems, this status is very much diminished, and the second 
law is revealed as a statement only about what is likely, within a framework of rules 
governing the evolution of probability that explicitly break time reversal symmetry. 
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